2024-05-04T12:04:13.817 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-04T12:04:13.817 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev} --verbose --archive /home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278 --name yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278/orig.config.yaml 2024-05-04T12:04:13.863 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-04T12:04:13.983 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278 branch: wip-yuri4-testing-2024-04-29-0642 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev} email: null first_in_suite: false job_id: '7686278' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-yuri4-testing-2024-04-29-0642 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 09dbd6bba52d19ba471c33bc15c008e6ad158ea6 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 09dbd6bba52d19ba471c33bc15c008e6ad158ea6 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri4-testing-2024-04-29-0642 sha1: 09dbd6bba52d19ba471c33bc15c008e6ad158ea6 owner: scheduled_yuriw@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1311 sha1: 09dbd6bba52d19ba471c33bc15c008e6ad158ea6 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri4-testing-2024-04-29-0642 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_09dbd6bba52d19ba471c33bc15c008e6ad158ea6/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 09dbd6bba52d19ba471c33bc15c008e6ad158ea6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi052.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDPZERTOJNDZxL2XAJILk+M8iDgy2uK9e+uQScedQDF0HhdKaCNtClQgSVnFoUD/Cy1/bMoA8+vv4Y39DyglmFA= smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM9FpB/ROanvtTByjSaVIgOKZcrhO6T9+F6tFHY31AHl+5ybolof2Jmzat+7DOV3/ui3LkwWN3sRAFnTRfk2HaY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-02_19:10:09 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-04T12:04:13.984 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_09dbd6bba52d19ba471c33bc15c008e6ad158ea6/qa; will attempt to use it 2024-05-04T12:04:13.984 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_09dbd6bba52d19ba471c33bc15c008e6ad158ea6/qa/tasks 2024-05-04T12:04:13.984 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-04T12:04:13.986 INFO:teuthology.task.internal:Checking packages... 2024-05-04T12:04:14.011 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '09dbd6bba52d19ba471c33bc15c008e6ad158ea6' 2024-05-04T12:04:14.011 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-04T12:04:14.011 INFO:teuthology.packaging:ref: None 2024-05-04T12:04:14.011 INFO:teuthology.packaging:tag: None 2024-05-04T12:04:14.011 INFO:teuthology.packaging:branch: wip-yuri4-testing-2024-04-29-0642 2024-05-04T12:04:14.011 INFO:teuthology.packaging:sha1: 09dbd6bba52d19ba471c33bc15c008e6ad158ea6 2024-05-04T12:04:14.011 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-yuri4-testing-2024-04-29-0642 2024-05-04T12:04:14.236 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2455-g09dbd6bb-1jammy 2024-05-04T12:04:14.238 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-04T12:04:14.244 INFO:teuthology.task.internal:no buildpackages task found 2024-05-04T12:04:14.245 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-04T12:04:14.296 INFO:teuthology.task.internal:Saving configuration 2024-05-04T12:04:14.313 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-04T12:04:14.318 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-04T12:04:14.351 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-04 11:58:19.653799', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILS3KvpDYPMNgfiEBjLuXOyIIpTRWKTeFjhReHtL9T/E'} 2024-05-04T12:04:14.381 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-04 11:58:19.652635', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIBVEF15bdJAvFo+wYWCVKh675AKaYGjUlxc6Of8/krn'} 2024-05-04T12:04:14.381 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-04T12:04:14.387 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-04T12:04:14.387 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-04T12:04:14.387 INFO:teuthology.run_tasks:Running task console_log... 2024-05-04T12:04:14.476 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f07cd1fd4c0>, signals=[15]) 2024-05-04T12:04:14.476 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-04T12:04:14.482 INFO:teuthology.task.internal:Opening connections... 2024-05-04T12:04:14.483 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T12:04:14.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T12:04:14.573 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T12:04:14.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T12:04:14.661 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-04T12:04:14.675 DEBUG:teuthology.orchestra.run.smithi052:> uname -m 2024-05-04T12:04:14.681 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2024-05-04T12:04:14.681 DEBUG:teuthology.orchestra.run.smithi052:> cat /etc/os-release 2024-05-04T12:04:14.726 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-04T12:04:14.726 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="Ubuntu" 2024-05-04T12:04:14.726 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="22.04" 2024-05-04T12:04:14.727 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-04T12:04:14.727 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_CODENAME=jammy 2024-05-04T12:04:14.727 INFO:teuthology.orchestra.run.smithi052.stdout:ID=ubuntu 2024-05-04T12:04:14.727 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE=debian 2024-05-04T12:04:14.727 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-04T12:04:14.727 INFO:teuthology.orchestra.run.smithi052.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-04T12:04:14.727 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-04T12:04:14.727 INFO:teuthology.orchestra.run.smithi052.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-04T12:04:14.727 INFO:teuthology.orchestra.run.smithi052.stdout:UBUNTU_CODENAME=jammy 2024-05-04T12:04:14.728 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2024-05-04T12:04:14.756 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2024-05-04T12:04:14.761 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2024-05-04T12:04:14.761 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2024-05-04T12:04:14.808 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-04T12:04:14.808 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="Ubuntu" 2024-05-04T12:04:14.808 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="22.04" 2024-05-04T12:04:14.808 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-04T12:04:14.808 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_CODENAME=jammy 2024-05-04T12:04:14.808 INFO:teuthology.orchestra.run.smithi111.stdout:ID=ubuntu 2024-05-04T12:04:14.808 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE=debian 2024-05-04T12:04:14.808 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-04T12:04:14.808 INFO:teuthology.orchestra.run.smithi111.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-04T12:04:14.808 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-04T12:04:14.808 INFO:teuthology.orchestra.run.smithi111.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-04T12:04:14.808 INFO:teuthology.orchestra.run.smithi111.stdout:UBUNTU_CODENAME=jammy 2024-05-04T12:04:14.809 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2024-05-04T12:04:14.848 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-04T12:04:14.857 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-04T12:04:14.862 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-04T12:04:14.863 DEBUG:teuthology.orchestra.run.smithi052:> test '!' -e /home/ubuntu/cephtest 2024-05-04T12:04:14.865 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2024-05-04T12:04:14.869 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-04T12:04:14.874 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-04T12:04:14.875 DEBUG:teuthology.orchestra.run.smithi052:> test -z $(ls -A /var/lib/ceph) 2024-05-04T12:04:14.911 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2024-05-04T12:04:14.926 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-04T12:04:14.999 INFO:teuthology.run_tasks:Running task kernel... 2024-05-04T12:04:15.018 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-04T12:04:15.018 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-04T12:04:15.019 DEBUG:teuthology.orchestra.run.smithi052:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-04T12:04:15.019 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-04T12:04:15.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T12:04:15.024 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-05-04T12:04:15.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T12:04:15.025 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-05-04T12:04:15.070 INFO:teuthology.orchestra.run.smithi052.stdout:5.15.0-56-generic 2024-05-04T12:04:15.070 INFO:teuthology.task.kernel:Running kernel on smithi052: 5.15.0-56-generic 2024-05-04T12:04:15.071 DEBUG:teuthology.orchestra.run.smithi052:> sudo apt-get clean 2024-05-04T12:04:15.072 INFO:teuthology.orchestra.run.smithi111.stdout:5.15.0-56-generic 2024-05-04T12:04:15.072 INFO:teuthology.task.kernel:Running kernel on smithi111: 5.15.0-56-generic 2024-05-04T12:04:15.072 DEBUG:teuthology.orchestra.run.smithi111:> sudo apt-get clean 2024-05-04T12:04:15.223 DEBUG:teuthology.orchestra.run.smithi052:> sudo apt-get update 2024-05-04T12:04:15.237 DEBUG:teuthology.orchestra.run.smithi111:> sudo apt-get update 2024-05-04T12:04:15.388 INFO:teuthology.orchestra.run.smithi052.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-04T12:04:15.388 INFO:teuthology.orchestra.run.smithi052.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-05-04T12:04:15.388 INFO:teuthology.orchestra.run.smithi052.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-05-04T12:04:15.389 INFO:teuthology.orchestra.run.smithi111.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-05-04T12:04:15.477 INFO:teuthology.orchestra.run.smithi052.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-05-04T12:04:15.497 INFO:teuthology.orchestra.run.smithi111.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-04T12:04:15.579 INFO:teuthology.orchestra.run.smithi111.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-05-04T12:04:15.617 INFO:teuthology.orchestra.run.smithi111.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [453 kB] 2024-05-04T12:04:15.633 INFO:teuthology.orchestra.run.smithi052.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [453 kB] 2024-05-04T12:04:15.702 INFO:teuthology.orchestra.run.smithi111.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-05-04T12:04:15.717 INFO:teuthology.orchestra.run.smithi052.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-05-04T12:04:15.717 INFO:teuthology.orchestra.run.smithi052.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [620 kB] 2024-05-04T12:04:15.748 INFO:teuthology.orchestra.run.smithi111.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-05-04T12:04:15.752 INFO:teuthology.orchestra.run.smithi111.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-05-04T12:04:15.752 INFO:teuthology.orchestra.run.smithi111.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-05-04T12:04:15.768 INFO:teuthology.orchestra.run.smithi052.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-05-04T12:04:15.770 INFO:teuthology.orchestra.run.smithi052.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-05-04T12:04:15.771 INFO:teuthology.orchestra.run.smithi052.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-05-04T12:04:15.790 INFO:teuthology.orchestra.run.smithi111.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-05-04T12:04:15.790 INFO:teuthology.orchestra.run.smithi111.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-05-04T12:04:15.795 INFO:teuthology.orchestra.run.smithi111.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-05-04T12:04:15.795 INFO:teuthology.orchestra.run.smithi111.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-05-04T12:04:15.804 INFO:teuthology.orchestra.run.smithi111.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-05-04T12:04:15.819 INFO:teuthology.orchestra.run.smithi111.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-05-04T12:04:15.820 INFO:teuthology.orchestra.run.smithi052.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-05-04T12:04:15.821 INFO:teuthology.orchestra.run.smithi052.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-05-04T12:04:15.822 INFO:teuthology.orchestra.run.smithi111.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-05-04T12:04:15.826 INFO:teuthology.orchestra.run.smithi052.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-05-04T12:04:15.826 INFO:teuthology.orchestra.run.smithi052.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-05-04T12:04:15.834 INFO:teuthology.orchestra.run.smithi052.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,612 kB] 2024-05-04T12:04:15.836 INFO:teuthology.orchestra.run.smithi111.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-05-04T12:04:15.837 INFO:teuthology.orchestra.run.smithi111.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-05-04T12:04:15.838 INFO:teuthology.orchestra.run.smithi052.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-05-04T12:04:15.855 INFO:teuthology.orchestra.run.smithi052.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-05-04T12:04:15.858 INFO:teuthology.orchestra.run.smithi052.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-05-04T12:04:15.861 INFO:teuthology.orchestra.run.smithi052.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-05-04T12:04:15.880 INFO:teuthology.orchestra.run.smithi111.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-05-04T12:04:15.880 INFO:teuthology.orchestra.run.smithi111.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-05-04T12:04:15.885 INFO:teuthology.orchestra.run.smithi052.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-05-04T12:04:15.885 INFO:teuthology.orchestra.run.smithi052.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-05-04T12:04:15.885 INFO:teuthology.orchestra.run.smithi052.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-05-04T12:04:15.898 INFO:teuthology.orchestra.run.smithi052.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [304 kB] 2024-05-04T12:04:15.906 INFO:teuthology.orchestra.run.smithi052.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-05-04T12:04:15.907 INFO:teuthology.orchestra.run.smithi052.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-05-04T12:04:15.942 INFO:teuthology.orchestra.run.smithi052.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-05-04T12:04:15.944 INFO:teuthology.orchestra.run.smithi052.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-05-04T12:04:15.946 INFO:teuthology.orchestra.run.smithi111.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-05-04T12:04:15.950 INFO:teuthology.orchestra.run.smithi052.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-05-04T12:04:15.950 INFO:teuthology.orchestra.run.smithi052.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [701 kB] 2024-05-04T12:04:15.960 INFO:teuthology.orchestra.run.smithi052.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,072 kB] 2024-05-04T12:04:15.972 INFO:teuthology.orchestra.run.smithi052.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-05-04T12:04:15.975 INFO:teuthology.orchestra.run.smithi052.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-05-04T12:04:15.975 INFO:teuthology.orchestra.run.smithi052.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-05-04T12:04:15.976 INFO:teuthology.orchestra.run.smithi052.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-05-04T12:04:15.976 INFO:teuthology.orchestra.run.smithi052.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-05-04T12:04:15.977 INFO:teuthology.orchestra.run.smithi052.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-05-04T12:04:15.991 INFO:teuthology.orchestra.run.smithi052.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-05-04T12:04:15.992 INFO:teuthology.orchestra.run.smithi052.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-05-04T12:04:15.993 INFO:teuthology.orchestra.run.smithi052.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-05-04T12:04:16.012 INFO:teuthology.orchestra.run.smithi052.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-05-04T12:04:16.012 INFO:teuthology.orchestra.run.smithi052.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-05-04T12:04:16.013 INFO:teuthology.orchestra.run.smithi052.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-05-04T12:04:16.013 INFO:teuthology.orchestra.run.smithi052.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-05-04T12:04:16.013 INFO:teuthology.orchestra.run.smithi052.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-05-04T12:04:16.081 INFO:teuthology.orchestra.run.smithi111.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [620 kB] 2024-05-04T12:04:16.281 INFO:teuthology.orchestra.run.smithi111.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,612 kB] 2024-05-04T12:04:16.431 INFO:teuthology.orchestra.run.smithi111.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [304 kB] 2024-05-04T12:04:16.446 INFO:teuthology.orchestra.run.smithi111.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-05-04T12:04:16.447 INFO:teuthology.orchestra.run.smithi111.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-05-04T12:04:16.448 INFO:teuthology.orchestra.run.smithi111.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-05-04T12:04:16.510 INFO:teuthology.orchestra.run.smithi111.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-05-04T12:04:16.517 INFO:teuthology.orchestra.run.smithi111.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-05-04T12:04:16.682 INFO:teuthology.orchestra.run.smithi111.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [701 kB] 2024-05-04T12:04:17.266 INFO:teuthology.orchestra.run.smithi111.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,072 kB] 2024-05-04T12:04:17.377 INFO:teuthology.orchestra.run.smithi111.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-05-04T12:04:17.397 INFO:teuthology.orchestra.run.smithi111.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-05-04T12:04:17.398 INFO:teuthology.orchestra.run.smithi111.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-05-04T12:04:17.399 INFO:teuthology.orchestra.run.smithi111.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-05-04T12:04:17.403 INFO:teuthology.orchestra.run.smithi111.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-05-04T12:04:17.403 INFO:teuthology.orchestra.run.smithi111.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-05-04T12:04:17.403 INFO:teuthology.orchestra.run.smithi111.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-05-04T12:04:17.407 INFO:teuthology.orchestra.run.smithi111.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-05-04T12:04:17.412 INFO:teuthology.orchestra.run.smithi111.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-05-04T12:04:17.576 INFO:teuthology.orchestra.run.smithi111.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-05-04T12:04:17.659 INFO:teuthology.orchestra.run.smithi111.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-05-04T12:04:17.741 INFO:teuthology.orchestra.run.smithi111.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-05-04T12:04:17.795 INFO:teuthology.orchestra.run.smithi111.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-05-04T12:04:17.845 INFO:teuthology.orchestra.run.smithi111.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-05-04T12:04:22.076 INFO:teuthology.orchestra.run.smithi111.stdout:Fetched 13.3 MB in 3s (5,227 kB/s) 2024-05-04T12:04:22.170 INFO:teuthology.orchestra.run.smithi052.stdout:Fetched 13.3 MB in 2s (6,782 kB/s) 2024-05-04T12:04:23.467 INFO:teuthology.orchestra.run.smithi111.stdout:Reading package lists... 2024-05-04T12:04:23.485 DEBUG:teuthology.orchestra.run.smithi111:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-04T12:04:23.545 INFO:teuthology.orchestra.run.smithi111.stdout:Reading package lists... 2024-05-04T12:04:23.723 INFO:teuthology.orchestra.run.smithi111.stdout:Building dependency tree... 2024-05-04T12:04:23.723 INFO:teuthology.orchestra.run.smithi111.stdout:Reading state information... 2024-05-04T12:04:23.878 INFO:teuthology.orchestra.run.smithi111.stdout:The following packages were automatically installed and are no longer required: 2024-05-04T12:04:23.878 INFO:teuthology.orchestra.run.smithi111.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-04T12:04:23.878 INFO:teuthology.orchestra.run.smithi111.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-04T12:04:23.879 INFO:teuthology.orchestra.run.smithi111.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-04T12:04:23.879 INFO:teuthology.orchestra.run.smithi111.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-05-04T12:04:23.879 INFO:teuthology.orchestra.run.smithi111.stdout: python2.7-minimal 2024-05-04T12:04:23.879 INFO:teuthology.orchestra.run.smithi111.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-04T12:04:23.880 INFO:teuthology.orchestra.run.smithi111.stdout:The following additional packages will be installed: 2024-05-04T12:04:23.880 INFO:teuthology.orchestra.run.smithi111.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-05-04T12:04:23.880 INFO:teuthology.orchestra.run.smithi111.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-05-04T12:04:23.880 INFO:teuthology.orchestra.run.smithi111.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-05-04T12:04:23.882 INFO:teuthology.orchestra.run.smithi111.stdout:Suggested packages: 2024-05-04T12:04:23.882 INFO:teuthology.orchestra.run.smithi111.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-05-04T12:04:23.882 INFO:teuthology.orchestra.run.smithi111.stdout:Recommended packages: 2024-05-04T12:04:23.882 INFO:teuthology.orchestra.run.smithi111.stdout: thermald 2024-05-04T12:04:23.905 INFO:teuthology.orchestra.run.smithi052.stdout:Reading package lists... 2024-05-04T12:04:23.923 DEBUG:teuthology.orchestra.run.smithi052:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-04T12:04:23.926 INFO:teuthology.orchestra.run.smithi111.stdout:The following NEW packages will be installed: 2024-05-04T12:04:23.926 INFO:teuthology.orchestra.run.smithi111.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-05-04T12:04:23.926 INFO:teuthology.orchestra.run.smithi111.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-05-04T12:04:23.926 INFO:teuthology.orchestra.run.smithi111.stdout: linux-modules-extra-5.15.0-105-generic 2024-05-04T12:04:23.928 INFO:teuthology.orchestra.run.smithi111.stdout:The following packages will be upgraded: 2024-05-04T12:04:23.928 INFO:teuthology.orchestra.run.smithi111.stdout: linux-generic linux-headers-generic linux-image-generic 2024-05-04T12:04:23.985 INFO:teuthology.orchestra.run.smithi052.stdout:Reading package lists... 2024-05-04T12:04:24.008 INFO:teuthology.orchestra.run.smithi111.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-05-04T12:04:24.008 INFO:teuthology.orchestra.run.smithi111.stdout:Need to get 113 MB of archives. 2024-05-04T12:04:24.008 INFO:teuthology.orchestra.run.smithi111.stdout:After this operation, 583 MB of additional disk space will be used. 2024-05-04T12:04:24.008 INFO:teuthology.orchestra.run.smithi111.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-05-04T12:04:24.190 INFO:teuthology.orchestra.run.smithi052.stdout:Building dependency tree... 2024-05-04T12:04:24.190 INFO:teuthology.orchestra.run.smithi052.stdout:Reading state information... 2024-05-04T12:04:24.354 INFO:teuthology.orchestra.run.smithi052.stdout:The following packages were automatically installed and are no longer required: 2024-05-04T12:04:24.354 INFO:teuthology.orchestra.run.smithi052.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-04T12:04:24.354 INFO:teuthology.orchestra.run.smithi052.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-04T12:04:24.355 INFO:teuthology.orchestra.run.smithi052.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-04T12:04:24.355 INFO:teuthology.orchestra.run.smithi052.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-05-04T12:04:24.355 INFO:teuthology.orchestra.run.smithi052.stdout: python2.7-minimal 2024-05-04T12:04:24.355 INFO:teuthology.orchestra.run.smithi052.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-04T12:04:24.356 INFO:teuthology.orchestra.run.smithi052.stdout:The following additional packages will be installed: 2024-05-04T12:04:24.356 INFO:teuthology.orchestra.run.smithi052.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-05-04T12:04:24.356 INFO:teuthology.orchestra.run.smithi052.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-05-04T12:04:24.356 INFO:teuthology.orchestra.run.smithi052.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-05-04T12:04:24.358 INFO:teuthology.orchestra.run.smithi052.stdout:Suggested packages: 2024-05-04T12:04:24.358 INFO:teuthology.orchestra.run.smithi052.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-05-04T12:04:24.358 INFO:teuthology.orchestra.run.smithi052.stdout:Recommended packages: 2024-05-04T12:04:24.358 INFO:teuthology.orchestra.run.smithi052.stdout: thermald 2024-05-04T12:04:24.412 INFO:teuthology.orchestra.run.smithi052.stdout:The following NEW packages will be installed: 2024-05-04T12:04:24.412 INFO:teuthology.orchestra.run.smithi052.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-05-04T12:04:24.412 INFO:teuthology.orchestra.run.smithi052.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-05-04T12:04:24.413 INFO:teuthology.orchestra.run.smithi052.stdout: linux-modules-extra-5.15.0-105-generic 2024-05-04T12:04:24.414 INFO:teuthology.orchestra.run.smithi052.stdout:The following packages will be upgraded: 2024-05-04T12:04:24.414 INFO:teuthology.orchestra.run.smithi052.stdout: linux-generic linux-headers-generic linux-image-generic 2024-05-04T12:04:24.464 INFO:teuthology.orchestra.run.smithi111.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-05-04T12:04:24.479 INFO:teuthology.orchestra.run.smithi052.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-05-04T12:04:24.479 INFO:teuthology.orchestra.run.smithi052.stdout:Need to get 113 MB of archives. 2024-05-04T12:04:24.479 INFO:teuthology.orchestra.run.smithi052.stdout:After this operation, 583 MB of additional disk space will be used. 2024-05-04T12:04:24.479 INFO:teuthology.orchestra.run.smithi052.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-05-04T12:04:24.575 INFO:teuthology.orchestra.run.smithi111.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-05-04T12:04:25.200 INFO:teuthology.orchestra.run.smithi052.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-05-04T12:04:25.464 INFO:teuthology.orchestra.run.smithi052.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-05-04T12:04:26.144 INFO:teuthology.orchestra.run.smithi111.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-05-04T12:04:26.144 INFO:teuthology.orchestra.run.smithi111.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-05-04T12:04:26.144 INFO:teuthology.orchestra.run.smithi111.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-05-04T12:04:26.376 INFO:teuthology.orchestra.run.smithi111.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-05-04T12:04:26.427 INFO:teuthology.orchestra.run.smithi111.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-05-04T12:04:26.548 INFO:teuthology.orchestra.run.smithi052.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-05-04T12:04:26.548 INFO:teuthology.orchestra.run.smithi052.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-05-04T12:04:26.548 INFO:teuthology.orchestra.run.smithi052.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-05-04T12:04:26.787 INFO:teuthology.orchestra.run.smithi052.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-05-04T12:04:26.809 INFO:teuthology.orchestra.run.smithi111.stdout:Fetched 113 MB in 2s (45.4 MB/s) 2024-05-04T12:04:26.821 INFO:teuthology.orchestra.run.smithi052.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-05-04T12:04:26.971 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-05-04T12:04:27.233 INFO:teuthology.orchestra.run.smithi052.stdout:Fetched 113 MB in 2s (47.2 MB/s) 2024-05-04T12:04:27.447 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-05-04T12:04:28.995 INFO:teuthology.orchestra.run.smithi111.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-05-04T12:04:28.999 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-05-04T12:04:29.102 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-05-04T12:04:29.496 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-05-04T12:04:29.795 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:31.246 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-05-04T12:04:31.267 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-05-04T12:04:31.387 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:31.824 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-05-04T12:04:31.845 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-05-04T12:04:31.874 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:31.913 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-05-04T12:04:31.935 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-05-04T12:04:32.035 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:32.482 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-05-04T12:04:32.504 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-05-04T12:04:32.532 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:38.809 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-05-04T12:04:38.935 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-05-04T12:04:39.260 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-05-04T12:04:39.381 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-05-04T12:04:39.669 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-05-04T12:04:39.692 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-05-04T12:04:39.715 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-05-04T12:04:39.759 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-05-04T12:04:39.786 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-05-04T12:04:40.129 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-05-04T12:04:40.265 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-05-04T12:04:40.585 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-05-04T12:04:40.607 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-05-04T12:04:40.651 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-05-04T12:04:45.749 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-05-04T12:04:45.786 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-05-04T12:04:45.824 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:46.836 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-05-04T12:04:46.874 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-05-04T12:04:46.911 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:48.503 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-05-04T12:04:48.628 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-05-04T12:04:48.953 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-05-04T12:04:49.075 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:49.201 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-05-04T12:04:49.354 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:49.806 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-05-04T12:04:49.956 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-05-04T12:04:50.314 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-05-04T12:04:50.419 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:50.545 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-05-04T12:04:50.653 INFO:teuthology.orchestra.run.smithi111.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-05-04T12:04:50.653 INFO:teuthology.orchestra.run.smithi111.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-05-04T12:04:50.653 INFO:teuthology.orchestra.run.smithi111.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-05-04T12:04:50.653 INFO:teuthology.orchestra.run.smithi111.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-05-04T12:04:50.924 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:51.624 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:52.891 INFO:teuthology.orchestra.run.smithi052.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-05-04T12:04:52.891 INFO:teuthology.orchestra.run.smithi052.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-05-04T12:04:52.891 INFO:teuthology.orchestra.run.smithi052.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-05-04T12:04:52.891 INFO:teuthology.orchestra.run.smithi052.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-05-04T12:04:53.010 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:53.087 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:54.410 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-05-04T12:04:54.538 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:54.545 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-05-04T12:04:54.671 INFO:teuthology.orchestra.run.smithi111.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:54.733 INFO:teuthology.orchestra.run.smithi111.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-05-04T12:04:54.733 INFO:teuthology.orchestra.run.smithi111.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-05-04T12:04:55.964 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-05-04T12:04:56.090 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-05-04T12:04:56.216 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-05-04T12:04:56.278 INFO:teuthology.orchestra.run.smithi052.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-05-04T12:04:56.278 INFO:teuthology.orchestra.run.smithi052.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-05-04T12:05:09.191 INFO:teuthology.orchestra.run.smithi111.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-05-04T12:05:09.191 INFO:teuthology.orchestra.run.smithi111.stdout:Sourcing file `/etc/default/grub' 2024-05-04T12:05:09.228 INFO:teuthology.orchestra.run.smithi111.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-04T12:05:09.249 INFO:teuthology.orchestra.run.smithi111.stdout:Generating grub configuration file ... 2024-05-04T12:05:09.501 INFO:teuthology.orchestra.run.smithi111.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-05-04T12:05:09.523 INFO:teuthology.orchestra.run.smithi111.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-05-04T12:05:09.815 INFO:teuthology.orchestra.run.smithi111.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-04T12:05:09.820 INFO:teuthology.orchestra.run.smithi111.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-04T12:05:09.891 INFO:teuthology.orchestra.run.smithi111.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-04T12:05:09.896 INFO:teuthology.orchestra.run.smithi111.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-04T12:05:10.076 INFO:teuthology.orchestra.run.smithi111.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-04T12:05:10.076 INFO:teuthology.orchestra.run.smithi111.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-05-04T12:05:10.076 INFO:teuthology.orchestra.run.smithi111.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-04T12:05:10.119 INFO:teuthology.orchestra.run.smithi111.stdout:done 2024-05-04T12:05:10.523 DEBUG:teuthology.orchestra.run.smithi111:> dpkg -s linux-image-generic 2024-05-04T12:05:10.545 INFO:teuthology.orchestra.run.smithi111.stdout:Package: linux-image-generic 2024-05-04T12:05:10.545 INFO:teuthology.orchestra.run.smithi111.stdout:Status: install ok installed 2024-05-04T12:05:10.545 INFO:teuthology.orchestra.run.smithi111.stdout:Priority: optional 2024-05-04T12:05:10.545 INFO:teuthology.orchestra.run.smithi111.stdout:Section: kernel 2024-05-04T12:05:10.545 INFO:teuthology.orchestra.run.smithi111.stdout:Installed-Size: 21 2024-05-04T12:05:10.545 INFO:teuthology.orchestra.run.smithi111.stdout:Maintainer: Ubuntu Kernel Team 2024-05-04T12:05:10.545 INFO:teuthology.orchestra.run.smithi111.stdout:Architecture: amd64 2024-05-04T12:05:10.545 INFO:teuthology.orchestra.run.smithi111.stdout:Source: linux-meta 2024-05-04T12:05:10.545 INFO:teuthology.orchestra.run.smithi111.stdout:Version: 5.15.0.105.102 2024-05-04T12:05:10.545 INFO:teuthology.orchestra.run.smithi111.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-04T12:05:10.546 INFO:teuthology.orchestra.run.smithi111.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-04T12:05:10.546 INFO:teuthology.orchestra.run.smithi111.stdout:Recommends: thermald 2024-05-04T12:05:10.546 INFO:teuthology.orchestra.run.smithi111.stdout:Description: Generic Linux kernel image 2024-05-04T12:05:10.546 INFO:teuthology.orchestra.run.smithi111.stdout: This package will always depend on the latest generic kernel image 2024-05-04T12:05:10.546 INFO:teuthology.orchestra.run.smithi111.stdout: available. 2024-05-04T12:05:10.546 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-05-04T12:05:10.546 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-04T12:05:10.546 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-04T12:05:10.546 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-04T12:05:10.547 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi111.front.sepia.ceph.com, path=None, version=distro) 2024-05-04T12:05:10.547 DEBUG:teuthology.orchestra.run.smithi111:> sudo apt-get clean 2024-05-04T12:05:10.778 DEBUG:teuthology.orchestra.run.smithi111:> sudo apt-get update 2024-05-04T12:05:10.903 INFO:teuthology.orchestra.run.smithi111.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-04T12:05:10.920 INFO:teuthology.orchestra.run.smithi111.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-04T12:05:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-04T12:05:11.018 INFO:teuthology.orchestra.run.smithi111.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-04T12:05:11.442 INFO:teuthology.orchestra.run.smithi052.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-05-04T12:05:11.442 INFO:teuthology.orchestra.run.smithi052.stdout:Sourcing file `/etc/default/grub' 2024-05-04T12:05:11.458 INFO:teuthology.orchestra.run.smithi052.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-04T12:05:11.483 INFO:teuthology.orchestra.run.smithi052.stdout:Generating grub configuration file ... 2024-05-04T12:05:11.897 INFO:teuthology.orchestra.run.smithi052.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-05-04T12:05:11.920 INFO:teuthology.orchestra.run.smithi052.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-05-04T12:05:12.271 INFO:teuthology.orchestra.run.smithi111.stdout:Reading package lists... 2024-05-04T12:05:12.289 DEBUG:teuthology.orchestra.run.smithi111:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-04T12:05:12.330 INFO:teuthology.orchestra.run.smithi052.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-04T12:05:12.336 INFO:teuthology.orchestra.run.smithi052.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-04T12:05:12.340 INFO:teuthology.orchestra.run.smithi111.stdout:Reading package lists... 2024-05-04T12:05:12.414 INFO:teuthology.orchestra.run.smithi052.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-04T12:05:12.420 INFO:teuthology.orchestra.run.smithi052.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-04T12:05:12.517 INFO:teuthology.orchestra.run.smithi111.stdout:Building dependency tree... 2024-05-04T12:05:12.517 INFO:teuthology.orchestra.run.smithi111.stdout:Reading state information... 2024-05-04T12:05:12.593 INFO:teuthology.orchestra.run.smithi052.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-04T12:05:12.594 INFO:teuthology.orchestra.run.smithi052.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-05-04T12:05:12.594 INFO:teuthology.orchestra.run.smithi052.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-04T12:05:12.627 INFO:teuthology.orchestra.run.smithi052.stdout:done 2024-05-04T12:05:12.675 INFO:teuthology.orchestra.run.smithi111.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-05-04T12:05:12.675 INFO:teuthology.orchestra.run.smithi111.stdout:The following packages were automatically installed and are no longer required: 2024-05-04T12:05:12.675 INFO:teuthology.orchestra.run.smithi111.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-04T12:05:12.675 INFO:teuthology.orchestra.run.smithi111.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-04T12:05:12.675 INFO:teuthology.orchestra.run.smithi111.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-04T12:05:12.676 INFO:teuthology.orchestra.run.smithi111.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-04T12:05:12.676 INFO:teuthology.orchestra.run.smithi111.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-04T12:05:12.676 INFO:teuthology.orchestra.run.smithi111.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-04T12:05:12.716 INFO:teuthology.orchestra.run.smithi111.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-05-04T12:05:12.718 DEBUG:teuthology.orchestra.run.smithi111:> dpkg -s linux-image-generic 2024-05-04T12:05:12.773 INFO:teuthology.orchestra.run.smithi111.stdout:Package: linux-image-generic 2024-05-04T12:05:12.774 INFO:teuthology.orchestra.run.smithi111.stdout:Status: install ok installed 2024-05-04T12:05:12.774 INFO:teuthology.orchestra.run.smithi111.stdout:Priority: optional 2024-05-04T12:05:12.774 INFO:teuthology.orchestra.run.smithi111.stdout:Section: kernel 2024-05-04T12:05:12.774 INFO:teuthology.orchestra.run.smithi111.stdout:Installed-Size: 21 2024-05-04T12:05:12.774 INFO:teuthology.orchestra.run.smithi111.stdout:Maintainer: Ubuntu Kernel Team 2024-05-04T12:05:12.774 INFO:teuthology.orchestra.run.smithi111.stdout:Architecture: amd64 2024-05-04T12:05:12.774 INFO:teuthology.orchestra.run.smithi111.stdout:Source: linux-meta 2024-05-04T12:05:12.774 INFO:teuthology.orchestra.run.smithi111.stdout:Version: 5.15.0.105.102 2024-05-04T12:05:12.774 INFO:teuthology.orchestra.run.smithi111.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-04T12:05:12.774 INFO:teuthology.orchestra.run.smithi111.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-04T12:05:12.774 INFO:teuthology.orchestra.run.smithi111.stdout:Recommends: thermald 2024-05-04T12:05:12.774 INFO:teuthology.orchestra.run.smithi111.stdout:Description: Generic Linux kernel image 2024-05-04T12:05:12.774 INFO:teuthology.orchestra.run.smithi111.stdout: This package will always depend on the latest generic kernel image 2024-05-04T12:05:12.774 INFO:teuthology.orchestra.run.smithi111.stdout: available. 2024-05-04T12:05:12.775 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2024-05-04T12:05:12.819 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.dxWlc8Crbu 2024-05-04T12:05:12.820 DEBUG:teuthology.orchestra.run.smithi111:> sudo cp /boot/grub/grub.cfg /tmp/tmp.dxWlc8Crbu 2024-05-04T12:05:12.875 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.dxWlc8Crbu 2024-05-04T12:05:13.000 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.dxWlc8Crbu is 10KB 2024-05-04T12:05:13.013 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.dxWlc8Crbu 2024-05-04T12:05:13.018 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-05-04T12:05:13.070 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:05:13.070 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-05-04T12:05:13.070 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-05-04T12:05:13.134 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-05-04T12:05:13.134 DEBUG:teuthology.orchestra.run.smithi111:> sudo update-grub 2024-05-04T12:05:13.191 DEBUG:teuthology.orchestra.run.smithi052:> dpkg -s linux-image-generic 2024-05-04T12:05:13.213 INFO:teuthology.orchestra.run.smithi052.stdout:Package: linux-image-generic 2024-05-04T12:05:13.214 INFO:teuthology.orchestra.run.smithi052.stdout:Status: install ok installed 2024-05-04T12:05:13.214 INFO:teuthology.orchestra.run.smithi052.stdout:Priority: optional 2024-05-04T12:05:13.214 INFO:teuthology.orchestra.run.smithi052.stdout:Section: kernel 2024-05-04T12:05:13.214 INFO:teuthology.orchestra.run.smithi052.stdout:Installed-Size: 21 2024-05-04T12:05:13.214 INFO:teuthology.orchestra.run.smithi052.stdout:Maintainer: Ubuntu Kernel Team 2024-05-04T12:05:13.214 INFO:teuthology.orchestra.run.smithi052.stdout:Architecture: amd64 2024-05-04T12:05:13.214 INFO:teuthology.orchestra.run.smithi052.stdout:Source: linux-meta 2024-05-04T12:05:13.214 INFO:teuthology.orchestra.run.smithi052.stdout:Version: 5.15.0.105.102 2024-05-04T12:05:13.214 INFO:teuthology.orchestra.run.smithi052.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-04T12:05:13.214 INFO:teuthology.orchestra.run.smithi052.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-04T12:05:13.214 INFO:teuthology.orchestra.run.smithi052.stdout:Recommends: thermald 2024-05-04T12:05:13.214 INFO:teuthology.orchestra.run.smithi052.stdout:Description: Generic Linux kernel image 2024-05-04T12:05:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: This package will always depend on the latest generic kernel image 2024-05-04T12:05:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: available. 2024-05-04T12:05:13.215 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-05-04T12:05:13.215 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-04T12:05:13.215 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-04T12:05:13.215 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-04T12:05:13.215 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi052.front.sepia.ceph.com, path=None, version=distro) 2024-05-04T12:05:13.215 DEBUG:teuthology.orchestra.run.smithi052:> sudo apt-get clean 2024-05-04T12:05:13.557 DEBUG:teuthology.orchestra.run.smithi052:> sudo apt-get update 2024-05-04T12:05:13.820 INFO:teuthology.orchestra.run.smithi052.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-04T12:05:13.820 INFO:teuthology.orchestra.run.smithi052.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-04T12:05:13.892 INFO:teuthology.orchestra.run.smithi052.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-04T12:05:13.973 INFO:teuthology.orchestra.run.smithi052.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-04T12:05:14.339 INFO:teuthology.orchestra.run.smithi111.stderr:Sourcing file `/etc/default/grub' 2024-05-04T12:05:14.340 INFO:teuthology.orchestra.run.smithi111.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-04T12:05:14.342 INFO:teuthology.orchestra.run.smithi111.stderr:Generating grub configuration file ... 2024-05-04T12:05:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-05-04T12:05:14.514 INFO:teuthology.orchestra.run.smithi111.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-05-04T12:05:14.723 INFO:teuthology.orchestra.run.smithi111.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-04T12:05:14.728 INFO:teuthology.orchestra.run.smithi111.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-04T12:05:14.768 INFO:teuthology.orchestra.run.smithi111.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-04T12:05:14.772 INFO:teuthology.orchestra.run.smithi111.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-04T12:05:14.886 INFO:teuthology.orchestra.run.smithi111.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-04T12:05:14.886 INFO:teuthology.orchestra.run.smithi111.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-04T12:05:14.886 INFO:teuthology.orchestra.run.smithi111.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-04T12:05:14.913 INFO:teuthology.orchestra.run.smithi111.stderr:done 2024-05-04T12:05:14.922 DEBUG:teuthology.orchestra.run.smithi111:> sudo shutdown -r now 2024-05-04T12:05:15.194 INFO:teuthology.orchestra.run.smithi052.stdout:Reading package lists... 2024-05-04T12:05:15.212 DEBUG:teuthology.orchestra.run.smithi052:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-04T12:05:15.277 INFO:teuthology.orchestra.run.smithi052.stdout:Reading package lists... 2024-05-04T12:05:15.494 INFO:teuthology.orchestra.run.smithi052.stdout:Building dependency tree... 2024-05-04T12:05:15.495 INFO:teuthology.orchestra.run.smithi052.stdout:Reading state information... 2024-05-04T12:05:15.677 INFO:teuthology.orchestra.run.smithi052.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-05-04T12:05:15.677 INFO:teuthology.orchestra.run.smithi052.stdout:The following packages were automatically installed and are no longer required: 2024-05-04T12:05:15.677 INFO:teuthology.orchestra.run.smithi052.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-04T12:05:15.677 INFO:teuthology.orchestra.run.smithi052.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-04T12:05:15.677 INFO:teuthology.orchestra.run.smithi052.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-04T12:05:15.678 INFO:teuthology.orchestra.run.smithi052.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-04T12:05:15.678 INFO:teuthology.orchestra.run.smithi052.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-04T12:05:15.678 INFO:teuthology.orchestra.run.smithi052.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-04T12:05:15.719 INFO:teuthology.orchestra.run.smithi052.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-05-04T12:05:15.721 DEBUG:teuthology.orchestra.run.smithi052:> dpkg -s linux-image-generic 2024-05-04T12:05:15.737 INFO:teuthology.orchestra.run.smithi052.stdout:Package: linux-image-generic 2024-05-04T12:05:15.737 INFO:teuthology.orchestra.run.smithi052.stdout:Status: install ok installed 2024-05-04T12:05:15.737 INFO:teuthology.orchestra.run.smithi052.stdout:Priority: optional 2024-05-04T12:05:15.738 INFO:teuthology.orchestra.run.smithi052.stdout:Section: kernel 2024-05-04T12:05:15.738 INFO:teuthology.orchestra.run.smithi052.stdout:Installed-Size: 21 2024-05-04T12:05:15.738 INFO:teuthology.orchestra.run.smithi052.stdout:Maintainer: Ubuntu Kernel Team 2024-05-04T12:05:15.738 INFO:teuthology.orchestra.run.smithi052.stdout:Architecture: amd64 2024-05-04T12:05:15.738 INFO:teuthology.orchestra.run.smithi052.stdout:Source: linux-meta 2024-05-04T12:05:15.738 INFO:teuthology.orchestra.run.smithi052.stdout:Version: 5.15.0.105.102 2024-05-04T12:05:15.738 INFO:teuthology.orchestra.run.smithi052.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-04T12:05:15.738 INFO:teuthology.orchestra.run.smithi052.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-04T12:05:15.738 INFO:teuthology.orchestra.run.smithi052.stdout:Recommends: thermald 2024-05-04T12:05:15.738 INFO:teuthology.orchestra.run.smithi052.stdout:Description: Generic Linux kernel image 2024-05-04T12:05:15.738 INFO:teuthology.orchestra.run.smithi052.stdout: This package will always depend on the latest generic kernel image 2024-05-04T12:05:15.738 INFO:teuthology.orchestra.run.smithi052.stdout: available. 2024-05-04T12:05:15.739 DEBUG:teuthology.orchestra.run.smithi052:> mktemp 2024-05-04T12:05:15.786 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.r4kODgCT2B 2024-05-04T12:05:15.786 DEBUG:teuthology.orchestra.run.smithi052:> sudo cp /boot/grub/grub.cfg /tmp/tmp.r4kODgCT2B 2024-05-04T12:05:15.843 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0666 /tmp/tmp.r4kODgCT2B 2024-05-04T12:05:15.970 DEBUG:teuthology.orchestra.remote:smithi052:/tmp/tmp.r4kODgCT2B is 10KB 2024-05-04T12:05:15.982 DEBUG:teuthology.orchestra.run.smithi052:> rm -fr /tmp/tmp.r4kODgCT2B 2024-05-04T12:05:15.988 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-05-04T12:05:16.041 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:05:16.041 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-05-04T12:05:16.041 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-05-04T12:05:16.131 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-05-04T12:05:16.131 DEBUG:teuthology.orchestra.run.smithi052:> sudo update-grub 2024-05-04T12:05:17.626 INFO:teuthology.orchestra.run.smithi052.stderr:Sourcing file `/etc/default/grub' 2024-05-04T12:05:17.627 INFO:teuthology.orchestra.run.smithi052.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-04T12:05:17.631 INFO:teuthology.orchestra.run.smithi052.stderr:Generating grub configuration file ... 2024-05-04T12:05:17.871 INFO:teuthology.orchestra.run.smithi052.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-05-04T12:05:17.877 INFO:teuthology.orchestra.run.smithi052.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-05-04T12:05:18.201 INFO:teuthology.orchestra.run.smithi052.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-04T12:05:18.207 INFO:teuthology.orchestra.run.smithi052.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-04T12:05:18.250 INFO:teuthology.orchestra.run.smithi052.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-04T12:05:18.255 INFO:teuthology.orchestra.run.smithi052.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-04T12:05:18.378 INFO:teuthology.orchestra.run.smithi052.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-04T12:05:18.378 INFO:teuthology.orchestra.run.smithi052.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-04T12:05:18.378 INFO:teuthology.orchestra.run.smithi052.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-04T12:05:18.396 INFO:teuthology.orchestra.run.smithi052.stderr:done 2024-05-04T12:05:18.397 DEBUG:teuthology.orchestra.run.smithi052:> sudo shutdown -r now 2024-05-04T12:05:44.950 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-04T12:05:44.951 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-04T12:05:44.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T12:05:48.446 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-04T12:05:48.446 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-05-04T12:05:48.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T12:06:03.294 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-05-04T12:06:06.878 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.52 2024-05-04T12:06:12.298 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-04T12:06:12.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T12:06:15.362 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-05-04T12:06:15.880 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-05-04T12:06:15.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T12:06:18.942 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.52 2024-05-04T12:06:27.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-04T12:06:27.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T12:06:30.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-05-04T12:06:30.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-05-04T12:06:30.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T12:06:45.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-04T12:06:45.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T12:07:02.724 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-05-04T12:07:03.256 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-05-04T12:07:03.256 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-05-04T12:07:03.256 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-05-04T12:07:03.304 INFO:teuthology.orchestra.run.smithi052.stdout:5.15.0-105-generic 2024-05-04T12:07:03.304 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-05-04T12:07:03.304 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-04T12:07:03.304 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-04T12:07:03.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-05-04T12:07:04.306 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-04T12:07:04.306 DEBUG:teuthology.orchestra.run.smithi052:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-04T12:07:04.388 INFO:teuthology.orchestra.run.smithi052.stdout:ttyS1 2024-05-04T12:07:04.409 DEBUG:teuthology.parallel:result is None 2024-05-04T12:07:21.967 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-04T12:07:21.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T12:07:22.273 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-05-04T12:07:22.821 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-04T12:07:22.821 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-05-04T12:07:22.822 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-05-04T12:07:22.870 INFO:teuthology.orchestra.run.smithi111.stdout:5.15.0-105-generic 2024-05-04T12:07:22.870 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-05-04T12:07:22.870 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-04T12:07:22.870 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-04T12:07:23.871 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-04T12:07:23.871 DEBUG:teuthology.orchestra.run.smithi111:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-04T12:07:23.965 INFO:teuthology.orchestra.run.smithi111.stdout:ttyS1 2024-05-04T12:07:23.992 DEBUG:teuthology.parallel:result is None 2024-05-04T12:07:23.992 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-04T12:07:23.999 INFO:teuthology.task.internal:Creating test directory... 2024-05-04T12:07:24.000 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-04T12:07:24.002 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-04T12:07:24.007 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-04T12:07:24.014 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-04T12:07:24.023 INFO:teuthology.task.internal:Creating archive directory... 2024-05-04T12:07:24.023 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-04T12:07:24.049 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-04T12:07:24.083 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-04T12:07:24.091 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-04T12:07:24.091 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-04T12:07:24.109 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-04T12:07:24.126 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T12:07:24.135 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T12:07:24.143 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T12:07:24.152 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T12:07:24.154 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-04T12:07:24.160 INFO:teuthology.task.internal:Configuring sudo... 2024-05-04T12:07:24.160 DEBUG:teuthology.orchestra.run.smithi052:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-04T12:07:24.181 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-04T12:07:24.210 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-04T12:07:24.219 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-04T12:07:24.219 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-04T12:07:24.233 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-04T12:07:24.262 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-04T12:07:24.285 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-04T12:07:24.332 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:07:24.333 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-04T12:07:24.397 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-04T12:07:24.403 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-04T12:07:24.450 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:07:24.450 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-04T12:07:24.507 DEBUG:teuthology.orchestra.run.smithi052:> sudo service rsyslog restart 2024-05-04T12:07:24.510 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2024-05-04T12:07:24.585 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-04T12:07:24.593 INFO:teuthology.task.internal:Starting timer... 2024-05-04T12:07:24.593 INFO:teuthology.run_tasks:Running task pcp... 2024-05-04T12:07:24.604 INFO:teuthology.run_tasks:Running task selinux... 2024-05-04T12:07:24.640 DEBUG:teuthology.task.selinux:Excluding smithi052: OS 'ubuntu' does not support SELinux 2024-05-04T12:07:24.640 DEBUG:teuthology.task.selinux:Excluding smithi111: OS 'ubuntu' does not support SELinux 2024-05-04T12:07:24.641 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-04T12:07:24.641 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-05-04T12:07:24.641 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-04T12:07:24.641 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-04T12:07:24.649 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-04T12:07:24.654 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-04T12:07:24.841 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-04T12:07:24.856 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-04T12:07:24.857 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi052.front.sepia.ceph.com,smithi111.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-04T12:14:28.219 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi052.front.sepia.ceph.com'), Remote(name='ubuntu@smithi111.front.sepia.ceph.com')] 2024-05-04T12:14:28.220 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-05-04T12:14:28.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T12:14:28.303 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-05-04T12:14:28.382 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-05-04T12:14:28.383 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-04T12:14:28.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T12:14:28.461 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-05-04T12:14:28.538 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-04T12:14:28.538 INFO:teuthology.run_tasks:Running task clock... 2024-05-04T12:14:28.549 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-04T12:14:28.549 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-04T12:14:28.549 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T12:14:28.552 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-04T12:14:28.552 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T12:14:28.579 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-04T12:14:28.579 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: Command line: ntpd -gq 2024-05-04T12:14:28.580 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: ---------------------------------------------------- 2024-05-04T12:14:28.580 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: ntp-4 is maintained by Network Time Foundation, 2024-05-04T12:14:28.580 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-04T12:14:28.580 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: corporation. Support and training for ntp-4 are 2024-05-04T12:14:28.580 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: available at https://www.nwtime.org/support 2024-05-04T12:14:28.580 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: ---------------------------------------------------- 2024-05-04T12:14:28.581 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: proto: precision = 0.045 usec (-24) 2024-05-04T12:14:28.581 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: basedate set to 2022-02-04 2024-05-04T12:14:28.581 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: gps base set to 2022-02-06 (week 2196) 2024-05-04T12:14:28.581 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-04T12:14:28.581 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: restrict ::: KOD does nothing without LIMITED. 2024-05-04T12:14:28.581 INFO:teuthology.orchestra.run.smithi052.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-04T12:14:28.581 INFO:teuthology.orchestra.run.smithi052.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-04T12:14:28.582 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: Listen and drop on 0 v6wildcard [::]:123 2024-05-04T12:14:28.582 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-04T12:14:28.582 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: Listen normally on 2 lo 127.0.0.1:123 2024-05-04T12:14:28.582 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: Listen normally on 3 ens1f0 172.21.15.52:123 2024-05-04T12:14:28.583 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: Listen normally on 4 lo [::1]:123 2024-05-04T12:14:28.583 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15de%4]:123 2024-05-04T12:14:28.583 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:28 ntpd[17169]: Listening on routing socket on fd #22 for interface updates 2024-05-04T12:14:28.611 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-04T12:14:28.611 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: Command line: ntpd -gq 2024-05-04T12:14:28.611 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: ---------------------------------------------------- 2024-05-04T12:14:28.611 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: ntp-4 is maintained by Network Time Foundation, 2024-05-04T12:14:28.611 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-04T12:14:28.611 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: corporation. Support and training for ntp-4 are 2024-05-04T12:14:28.611 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: available at https://www.nwtime.org/support 2024-05-04T12:14:28.611 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: ---------------------------------------------------- 2024-05-04T12:14:28.611 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: proto: precision = 0.043 usec (-24) 2024-05-04T12:14:28.611 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: basedate set to 2022-02-04 2024-05-04T12:14:28.611 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: gps base set to 2022-02-06 (week 2196) 2024-05-04T12:14:28.611 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-04T12:14:28.612 INFO:teuthology.orchestra.run.smithi111.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-04T12:14:28.612 INFO:teuthology.orchestra.run.smithi111.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-04T12:14:28.613 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: restrict ::: KOD does nothing without LIMITED. 2024-05-04T12:14:28.613 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: Listen and drop on 0 v6wildcard [::]:123 2024-05-04T12:14:28.613 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-04T12:14:28.613 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: Listen normally on 2 lo 127.0.0.1:123 2024-05-04T12:14:28.613 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: Listen normally on 3 enp3s0f1 172.21.15.111:123 2024-05-04T12:14:28.613 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: Listen normally on 4 lo [::1]:123 2024-05-04T12:14:28.613 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cc2f%5]:123 2024-05-04T12:14:28.613 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:28 ntpd[17262]: Listening on routing socket on fd #22 for interface updates 2024-05-04T12:14:29.580 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:29 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:29.580 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:29 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:29.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:29 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:29.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:29 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:29.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:29 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:29.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:29 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:29.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:29 ntpd[17169]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:29.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:29 ntpd[17169]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:29.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:29 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:29.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:29 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:29.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:29 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:29.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:29 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:29.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:29 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:29.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:29 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:30.580 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:30 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:30.580 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:30 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:30.580 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:30 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:30.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:30 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:30.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:30 ntpd[17169]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:30.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:30 ntpd[17169]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:30.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:30 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:30.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:30 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:30.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:30 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:30.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:30 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:30.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:30 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:30.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:30 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:31.580 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:31 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:31.580 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:31 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:31.580 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:31 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:31.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:31 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:31.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:31 ntpd[17169]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:31.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:31 ntpd[17169]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:31.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:31 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:31.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:31 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:31.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:31 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:31.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:31 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:31.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:31 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:31.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:31 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:32.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:32 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:32.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:32 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:32.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:32 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:32.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:32 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:32.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:32 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:32.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:32 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:32.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:32 ntpd[17169]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:32.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:32 ntpd[17169]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:32.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:32 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:32.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:32 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:32.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:32 ntpd[17169]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:32.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:32 ntpd[17169]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:32.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:32 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:32.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:32 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:32.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:32 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:32.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:32 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:32.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:32 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:32.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:32 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:33.580 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:33 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:33.580 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:33 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:33.580 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:33 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:33.580 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:33 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:33.580 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:33 ntpd[17169]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:33.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:33 ntpd[17169]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:33.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:33 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:33.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:33 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:33.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:33 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:33.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:33 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:33.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:33 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:33.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:33 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:33.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:33 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:33.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:33 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:33.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:33 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:33.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:33 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:33.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:33 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:33.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:33 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:33.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:33 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:33.613 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:33 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:34.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:34 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:34.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:34 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:34.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:34 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:34.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:34 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:34.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:34 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:34.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:34 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:34.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:34 ntpd[17169]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:34.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:34 ntpd[17169]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:34.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:34 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:34.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:34 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:34.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:34 ntpd[17169]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:34.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:34 ntpd[17169]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:34.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:34 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:34.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:34 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:34.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:34 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:34.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:34 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:34.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:34 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:34.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:34 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:35.580 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:35 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:35.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:35 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:35.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:35 ntpd[17169]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:35.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:35 ntpd[17169]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:35.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:35 ntpd[17169]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:35.581 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:35 ntpd[17169]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:35.581 INFO:teuthology.orchestra.run.smithi052.stdout: 4 May 12:14:35 ntpd[17169]: ntpd: time slew -0.000090 s 2024-05-04T12:14:35.581 INFO:teuthology.orchestra.run.smithi052.stdout:ntpd: time slew -0.000090s 2024-05-04T12:14:35.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:35 ntpd[17169]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-04T12:14:35.582 INFO:teuthology.orchestra.run.smithi052.stderr: 4 May 12:14:35 ntpd[17169]: can't open /var/log/ntpstats/loopstats.20240504: Permission denied 2024-05-04T12:14:35.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:35 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:35.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:35 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:35.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:35 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:35.611 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:35 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:35.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:35 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:35.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:35 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:35.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:35 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:35.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:35 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:35.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:35 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-04T12:14:35.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:35 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240504: Permission denied 2024-05-04T12:14:35.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:35 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-04T12:14:35.612 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:35 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240504: Permission denied 2024-05-04T12:14:35.612 INFO:teuthology.orchestra.run.smithi111.stdout: 4 May 12:14:35 ntpd[17262]: ntpd: time slew -0.002234 s 2024-05-04T12:14:35.612 INFO:teuthology.orchestra.run.smithi111.stdout:ntpd: time slew -0.002234s 2024-05-04T12:14:35.613 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:35 ntpd[17262]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-04T12:14:35.613 INFO:teuthology.orchestra.run.smithi111.stderr: 4 May 12:14:35 ntpd[17262]: can't open /var/log/ntpstats/loopstats.20240504: Permission denied 2024-05-04T12:14:35.634 INFO:teuthology.orchestra.run.smithi052.stdout: remote refid st t when poll reach delay offset jitter 2024-05-04T12:14:35.634 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================== 2024-05-04T12:14:35.634 INFO:teuthology.orchestra.run.smithi052.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-04T12:14:35.634 INFO:teuthology.orchestra.run.smithi052.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-04T12:14:35.634 INFO:teuthology.orchestra.run.smithi052.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-04T12:14:35.634 INFO:teuthology.orchestra.run.smithi052.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-04T12:14:35.679 INFO:teuthology.orchestra.run.smithi111.stdout: remote refid st t when poll reach delay offset jitter 2024-05-04T12:14:35.679 INFO:teuthology.orchestra.run.smithi111.stdout:============================================================================== 2024-05-04T12:14:35.679 INFO:teuthology.orchestra.run.smithi111.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-04T12:14:35.679 INFO:teuthology.orchestra.run.smithi111.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-04T12:14:35.680 INFO:teuthology.orchestra.run.smithi111.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-04T12:14:35.680 INFO:teuthology.orchestra.run.smithi111.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-04T12:14:35.680 INFO:teuthology.run_tasks:Running task install... 2024-05-04T12:14:35.691 DEBUG:teuthology.task.install:project ceph 2024-05-04T12:14:35.691 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '09dbd6bba52d19ba471c33bc15c008e6ad158ea6'}} 2024-05-04T12:14:35.691 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '09dbd6bba52d19ba471c33bc15c008e6ad158ea6'} 2024-05-04T12:14:35.691 INFO:teuthology.task.install:Using flavor: default 2024-05-04T12:14:35.697 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-04T12:14:35.697 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-04T12:14:35.698 DEBUG:teuthology.orchestra.run.smithi052:> sudo apt-key list | grep Ceph 2024-05-04T12:14:35.698 DEBUG:teuthology.orchestra.run.smithi111:> sudo apt-key list | grep Ceph 2024-05-04T12:14:35.757 INFO:teuthology.orchestra.run.smithi052.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-04T12:14:35.777 INFO:teuthology.orchestra.run.smithi111.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-04T12:14:35.787 INFO:teuthology.orchestra.run.smithi052.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-04T12:14:35.787 INFO:teuthology.orchestra.run.smithi052.stdout:uid [ unknown] Ceph.com (release key) 2024-05-04T12:14:35.788 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-04T12:14:35.788 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-04T12:14:35.789 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=09dbd6bba52d19ba471c33bc15c008e6ad158ea6 2024-05-04T12:14:35.805 INFO:teuthology.orchestra.run.smithi111.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-04T12:14:35.805 INFO:teuthology.orchestra.run.smithi111.stdout:uid [ unknown] Ceph.com (release key) 2024-05-04T12:14:35.805 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-04T12:14:35.805 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-04T12:14:35.805 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=09dbd6bba52d19ba471c33bc15c008e6ad158ea6 2024-05-04T12:14:35.934 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default/ 2024-05-04T12:14:35.935 INFO:teuthology.task.install.deb:Package version is 19.0.0-2455-g09dbd6bb-1jammy 2024-05-04T12:14:36.025 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default/ 2024-05-04T12:14:36.025 INFO:teuthology.task.install.deb:Package version is 19.0.0-2455-g09dbd6bb-1jammy 2024-05-04T12:14:36.103 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:14:36.103 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-04T12:14:36.118 DEBUG:teuthology.orchestra.run.smithi052:> sudo apt-get update 2024-05-04T12:14:36.192 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:14:36.192 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-04T12:14:36.206 DEBUG:teuthology.orchestra.run.smithi111:> sudo apt-get update 2024-05-04T12:14:36.376 INFO:teuthology.orchestra.run.smithi111.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-04T12:14:36.389 INFO:teuthology.orchestra.run.smithi111.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-04T12:14:36.408 INFO:teuthology.orchestra.run.smithi052.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy InRelease 2024-05-04T12:14:36.409 INFO:teuthology.orchestra.run.smithi111.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-04T12:14:36.420 INFO:teuthology.orchestra.run.smithi052.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-04T12:14:36.433 INFO:teuthology.orchestra.run.smithi052.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-04T12:14:36.444 INFO:teuthology.orchestra.run.smithi052.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy Release [6,978 B] 2024-05-04T12:14:36.482 INFO:teuthology.orchestra.run.smithi052.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-04T12:14:36.486 INFO:teuthology.orchestra.run.smithi111.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy InRelease 2024-05-04T12:14:36.490 INFO:teuthology.orchestra.run.smithi052.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-04T12:14:36.493 INFO:teuthology.orchestra.run.smithi111.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-04T12:14:36.518 INFO:teuthology.orchestra.run.smithi052.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,416 B] 2024-05-04T12:14:36.522 INFO:teuthology.orchestra.run.smithi111.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy Release [6,978 B] 2024-05-04T12:14:36.560 INFO:teuthology.orchestra.run.smithi111.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-04T12:14:36.561 INFO:teuthology.orchestra.run.smithi052.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-05-04T12:14:36.572 INFO:teuthology.orchestra.run.smithi052.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-04T12:14:36.595 INFO:teuthology.orchestra.run.smithi111.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,416 B] 2024-05-04T12:14:36.637 INFO:teuthology.orchestra.run.smithi111.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-05-04T12:14:36.713 INFO:teuthology.orchestra.run.smithi111.stdout:Fetched 28.2 kB in 0s (84.5 kB/s) 2024-05-04T12:14:36.815 INFO:teuthology.orchestra.run.smithi052.stdout:Fetched 28.2 kB in 1s (54.9 kB/s) 2024-05-04T12:14:37.647 INFO:teuthology.orchestra.run.smithi111.stdout:Reading package lists... 2024-05-04T12:14:37.662 DEBUG:teuthology.orchestra.run.smithi111:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2455-g09dbd6bb-1jammy cephadm=19.0.0-2455-g09dbd6bb-1jammy ceph-mds=19.0.0-2455-g09dbd6bb-1jammy ceph-mgr=19.0.0-2455-g09dbd6bb-1jammy ceph-common=19.0.0-2455-g09dbd6bb-1jammy ceph-fuse=19.0.0-2455-g09dbd6bb-1jammy ceph-test=19.0.0-2455-g09dbd6bb-1jammy ceph-volume=19.0.0-2455-g09dbd6bb-1jammy radosgw=19.0.0-2455-g09dbd6bb-1jammy python3-rados=19.0.0-2455-g09dbd6bb-1jammy python3-rgw=19.0.0-2455-g09dbd6bb-1jammy python3-cephfs=19.0.0-2455-g09dbd6bb-1jammy python3-rbd=19.0.0-2455-g09dbd6bb-1jammy libcephfs2=19.0.0-2455-g09dbd6bb-1jammy libcephfs-dev=19.0.0-2455-g09dbd6bb-1jammy librados2=19.0.0-2455-g09dbd6bb-1jammy librbd1=19.0.0-2455-g09dbd6bb-1jammy rbd-fuse=19.0.0-2455-g09dbd6bb-1jammy python3-cephfs=19.0.0-2455-g09dbd6bb-1jammy cephfs-shell=19.0.0-2455-g09dbd6bb-1jammy cephfs-top=19.0.0-2455-g09dbd6bb-1jammy cephfs-mirror=19.0.0-2455-g09dbd6bb-1jammy 2024-05-04T12:14:37.724 INFO:teuthology.orchestra.run.smithi111.stdout:Reading package lists... 2024-05-04T12:14:37.761 INFO:teuthology.orchestra.run.smithi052.stdout:Reading package lists... 2024-05-04T12:14:37.776 DEBUG:teuthology.orchestra.run.smithi052:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2455-g09dbd6bb-1jammy cephadm=19.0.0-2455-g09dbd6bb-1jammy ceph-mds=19.0.0-2455-g09dbd6bb-1jammy ceph-mgr=19.0.0-2455-g09dbd6bb-1jammy ceph-common=19.0.0-2455-g09dbd6bb-1jammy ceph-fuse=19.0.0-2455-g09dbd6bb-1jammy ceph-test=19.0.0-2455-g09dbd6bb-1jammy ceph-volume=19.0.0-2455-g09dbd6bb-1jammy radosgw=19.0.0-2455-g09dbd6bb-1jammy python3-rados=19.0.0-2455-g09dbd6bb-1jammy python3-rgw=19.0.0-2455-g09dbd6bb-1jammy python3-cephfs=19.0.0-2455-g09dbd6bb-1jammy python3-rbd=19.0.0-2455-g09dbd6bb-1jammy libcephfs2=19.0.0-2455-g09dbd6bb-1jammy libcephfs-dev=19.0.0-2455-g09dbd6bb-1jammy librados2=19.0.0-2455-g09dbd6bb-1jammy librbd1=19.0.0-2455-g09dbd6bb-1jammy rbd-fuse=19.0.0-2455-g09dbd6bb-1jammy python3-cephfs=19.0.0-2455-g09dbd6bb-1jammy cephfs-shell=19.0.0-2455-g09dbd6bb-1jammy cephfs-top=19.0.0-2455-g09dbd6bb-1jammy cephfs-mirror=19.0.0-2455-g09dbd6bb-1jammy 2024-05-04T12:14:37.838 INFO:teuthology.orchestra.run.smithi052.stdout:Reading package lists... 2024-05-04T12:14:37.929 INFO:teuthology.orchestra.run.smithi111.stdout:Building dependency tree... 2024-05-04T12:14:37.930 INFO:teuthology.orchestra.run.smithi111.stdout:Reading state information... 2024-05-04T12:14:38.041 INFO:teuthology.orchestra.run.smithi052.stdout:Building dependency tree... 2024-05-04T12:14:38.042 INFO:teuthology.orchestra.run.smithi052.stdout:Reading state information... 2024-05-04T12:14:38.125 INFO:teuthology.orchestra.run.smithi111.stdout:The following packages were automatically installed and are no longer required: 2024-05-04T12:14:38.125 INFO:teuthology.orchestra.run.smithi111.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-04T12:14:38.125 INFO:teuthology.orchestra.run.smithi111.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-04T12:14:38.125 INFO:teuthology.orchestra.run.smithi111.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-04T12:14:38.126 INFO:teuthology.orchestra.run.smithi111.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-04T12:14:38.126 INFO:teuthology.orchestra.run.smithi111.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-04T12:14:38.127 INFO:teuthology.orchestra.run.smithi111.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-04T12:14:38.127 INFO:teuthology.orchestra.run.smithi111.stdout:The following additional packages will be installed: 2024-05-04T12:14:38.127 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-04T12:14:38.127 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-04T12:14:38.127 INFO:teuthology.orchestra.run.smithi111.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-04T12:14:38.127 INFO:teuthology.orchestra.run.smithi111.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-04T12:14:38.128 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-05-04T12:14:38.128 INFO:teuthology.orchestra.run.smithi111.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-05-04T12:14:38.128 INFO:teuthology.orchestra.run.smithi111.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-04T12:14:38.128 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-04T12:14:38.128 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-04T12:14:38.128 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-05-04T12:14:38.128 INFO:teuthology.orchestra.run.smithi111.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-05-04T12:14:38.128 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-05-04T12:14:38.128 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-05-04T12:14:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-05-04T12:14:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-04T12:14:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-05-04T12:14:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-05-04T12:14:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-05-04T12:14:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-05-04T12:14:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-05-04T12:14:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-05-04T12:14:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-05-04T12:14:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-05-04T12:14:38.129 INFO:teuthology.orchestra.run.smithi111.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-05-04T12:14:38.130 INFO:teuthology.orchestra.run.smithi111.stdout:Suggested packages: 2024-05-04T12:14:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-05-04T12:14:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-05-04T12:14:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-05-04T12:14:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-05-04T12:14:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-05-04T12:14:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-04T12:14:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-05-04T12:14:38.130 INFO:teuthology.orchestra.run.smithi111.stdout:Recommended packages: 2024-05-04T12:14:38.130 INFO:teuthology.orchestra.run.smithi111.stdout: btrfs-tools 2024-05-04T12:14:38.237 INFO:teuthology.orchestra.run.smithi111.stdout:The following NEW packages will be installed: 2024-05-04T12:14:38.237 INFO:teuthology.orchestra.run.smithi111.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-04T12:14:38.237 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-04T12:14:38.237 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-04T12:14:38.237 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-05-04T12:14:38.237 INFO:teuthology.orchestra.run.smithi111.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-04T12:14:38.237 INFO:teuthology.orchestra.run.smithi111.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-05-04T12:14:38.237 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-05-04T12:14:38.239 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-04T12:14:38.240 INFO:teuthology.orchestra.run.smithi111.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-04T12:14:38.240 INFO:teuthology.orchestra.run.smithi111.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-04T12:14:38.240 INFO:teuthology.orchestra.run.smithi111.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-04T12:14:38.240 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-04T12:14:38.240 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-05-04T12:14:38.240 INFO:teuthology.orchestra.run.smithi111.stdout: socat unzip xmlstarlet zip 2024-05-04T12:14:38.246 INFO:teuthology.orchestra.run.smithi052.stdout:The following packages were automatically installed and are no longer required: 2024-05-04T12:14:38.246 INFO:teuthology.orchestra.run.smithi052.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-04T12:14:38.246 INFO:teuthology.orchestra.run.smithi052.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-04T12:14:38.246 INFO:teuthology.orchestra.run.smithi052.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-04T12:14:38.246 INFO:teuthology.orchestra.run.smithi052.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-04T12:14:38.247 INFO:teuthology.orchestra.run.smithi052.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-04T12:14:38.247 INFO:teuthology.orchestra.run.smithi052.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-04T12:14:38.247 INFO:teuthology.orchestra.run.smithi052.stdout:The following additional packages will be installed: 2024-05-04T12:14:38.247 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-04T12:14:38.248 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-04T12:14:38.248 INFO:teuthology.orchestra.run.smithi052.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-04T12:14:38.248 INFO:teuthology.orchestra.run.smithi052.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-04T12:14:38.248 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-05-04T12:14:38.250 INFO:teuthology.orchestra.run.smithi052.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-05-04T12:14:38.250 INFO:teuthology.orchestra.run.smithi052.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-04T12:14:38.250 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-04T12:14:38.250 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-04T12:14:38.250 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-05-04T12:14:38.250 INFO:teuthology.orchestra.run.smithi052.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-05-04T12:14:38.250 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-05-04T12:14:38.250 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-05-04T12:14:38.250 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-05-04T12:14:38.250 INFO:teuthology.orchestra.run.smithi052.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-04T12:14:38.251 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-05-04T12:14:38.251 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-05-04T12:14:38.251 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-05-04T12:14:38.251 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-05-04T12:14:38.251 INFO:teuthology.orchestra.run.smithi052.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-05-04T12:14:38.251 INFO:teuthology.orchestra.run.smithi052.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-05-04T12:14:38.251 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-05-04T12:14:38.251 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-05-04T12:14:38.251 INFO:teuthology.orchestra.run.smithi052.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-05-04T12:14:38.251 INFO:teuthology.orchestra.run.smithi052.stdout:Suggested packages: 2024-05-04T12:14:38.251 INFO:teuthology.orchestra.run.smithi052.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-05-04T12:14:38.252 INFO:teuthology.orchestra.run.smithi052.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-05-04T12:14:38.252 INFO:teuthology.orchestra.run.smithi052.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-05-04T12:14:38.252 INFO:teuthology.orchestra.run.smithi052.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-05-04T12:14:38.252 INFO:teuthology.orchestra.run.smithi052.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-05-04T12:14:38.252 INFO:teuthology.orchestra.run.smithi052.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-04T12:14:38.252 INFO:teuthology.orchestra.run.smithi052.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-05-04T12:14:38.252 INFO:teuthology.orchestra.run.smithi052.stdout:Recommended packages: 2024-05-04T12:14:38.252 INFO:teuthology.orchestra.run.smithi052.stdout: btrfs-tools 2024-05-04T12:14:38.302 INFO:teuthology.orchestra.run.smithi111.stdout:0 upgraded, 124 newly installed, 0 to remove and 303 not upgraded. 2024-05-04T12:14:38.303 INFO:teuthology.orchestra.run.smithi111.stdout:Need to get 180 MB of archives. 2024-05-04T12:14:38.303 INFO:teuthology.orchestra.run.smithi111.stdout:After this operation, 823 MB of additional disk space will be used. 2024-05-04T12:14:38.303 INFO:teuthology.orchestra.run.smithi111.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-05-04T12:14:38.322 INFO:teuthology.orchestra.run.smithi111.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-04T12:14:38.334 INFO:teuthology.orchestra.run.smithi111.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-05-04T12:14:38.362 INFO:teuthology.orchestra.run.smithi052.stdout:The following NEW packages will be installed: 2024-05-04T12:14:38.362 INFO:teuthology.orchestra.run.smithi052.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-04T12:14:38.362 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-04T12:14:38.362 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-04T12:14:38.362 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-05-04T12:14:38.364 INFO:teuthology.orchestra.run.smithi052.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-04T12:14:38.364 INFO:teuthology.orchestra.run.smithi052.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-05-04T12:14:38.364 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi052.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-04T12:14:38.366 INFO:teuthology.orchestra.run.smithi052.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-04T12:14:38.366 INFO:teuthology.orchestra.run.smithi052.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-04T12:14:38.366 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-04T12:14:38.366 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-05-04T12:14:38.366 INFO:teuthology.orchestra.run.smithi052.stdout: socat unzip xmlstarlet zip 2024-05-04T12:14:38.380 INFO:teuthology.orchestra.run.smithi111.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-04T12:14:38.380 INFO:teuthology.orchestra.run.smithi111.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2455-g09dbd6bb-1jammy [3,543 kB] 2024-05-04T12:14:38.404 INFO:teuthology.orchestra.run.smithi111.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-04T12:14:38.407 INFO:teuthology.orchestra.run.smithi111.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-04T12:14:38.420 INFO:teuthology.orchestra.run.smithi111.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-04T12:14:38.432 INFO:teuthology.orchestra.run.smithi052.stdout:0 upgraded, 124 newly installed, 0 to remove and 303 not upgraded. 2024-05-04T12:14:38.432 INFO:teuthology.orchestra.run.smithi052.stdout:Need to get 180 MB of archives. 2024-05-04T12:14:38.432 INFO:teuthology.orchestra.run.smithi052.stdout:After this operation, 823 MB of additional disk space will be used. 2024-05-04T12:14:38.432 INFO:teuthology.orchestra.run.smithi052.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-05-04T12:14:38.453 INFO:teuthology.orchestra.run.smithi052.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-04T12:14:38.464 INFO:teuthology.orchestra.run.smithi052.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-05-04T12:14:38.470 INFO:teuthology.orchestra.run.smithi111.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-04T12:14:38.475 INFO:teuthology.orchestra.run.smithi111.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-04T12:14:38.486 INFO:teuthology.orchestra.run.smithi111.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-04T12:14:38.489 INFO:teuthology.orchestra.run.smithi111.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-04T12:14:38.490 INFO:teuthology.orchestra.run.smithi111.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-04T12:14:38.498 INFO:teuthology.orchestra.run.smithi111.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-05-04T12:14:38.500 INFO:teuthology.orchestra.run.smithi111.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-05-04T12:14:38.502 INFO:teuthology.orchestra.run.smithi111.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-05-04T12:14:38.503 INFO:teuthology.orchestra.run.smithi111.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-05-04T12:14:38.503 INFO:teuthology.orchestra.run.smithi111.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-05-04T12:14:38.512 INFO:teuthology.orchestra.run.smithi052.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-04T12:14:38.513 INFO:teuthology.orchestra.run.smithi111.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-05-04T12:14:38.515 INFO:teuthology.orchestra.run.smithi111.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-05-04T12:14:38.517 INFO:teuthology.orchestra.run.smithi111.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-04T12:14:38.517 INFO:teuthology.orchestra.run.smithi111.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-04T12:14:38.532 INFO:teuthology.orchestra.run.smithi111.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-04T12:14:38.533 INFO:teuthology.orchestra.run.smithi111.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-04T12:14:38.533 INFO:teuthology.orchestra.run.smithi111.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-04T12:14:38.534 INFO:teuthology.orchestra.run.smithi111.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-04T12:14:38.534 INFO:teuthology.orchestra.run.smithi111.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-04T12:14:38.534 INFO:teuthology.orchestra.run.smithi111.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-04T12:14:38.537 INFO:teuthology.orchestra.run.smithi052.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-04T12:14:38.540 INFO:teuthology.orchestra.run.smithi052.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-04T12:14:38.552 INFO:teuthology.orchestra.run.smithi111.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-04T12:14:38.552 INFO:teuthology.orchestra.run.smithi111.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-04T12:14:38.554 INFO:teuthology.orchestra.run.smithi111.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-04T12:14:38.555 INFO:teuthology.orchestra.run.smithi052.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-04T12:14:38.555 INFO:teuthology.orchestra.run.smithi111.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-04T12:14:38.571 INFO:teuthology.orchestra.run.smithi111.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-04T12:14:38.572 INFO:teuthology.orchestra.run.smithi111.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-04T12:14:38.572 INFO:teuthology.orchestra.run.smithi111.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-04T12:14:38.573 INFO:teuthology.orchestra.run.smithi111.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-04T12:14:38.573 INFO:teuthology.orchestra.run.smithi111.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-04T12:14:38.574 INFO:teuthology.orchestra.run.smithi111.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-04T12:14:38.590 INFO:teuthology.orchestra.run.smithi052.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2455-g09dbd6bb-1jammy [3,543 kB] 2024-05-04T12:14:38.591 INFO:teuthology.orchestra.run.smithi111.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-04T12:14:38.591 INFO:teuthology.orchestra.run.smithi111.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-04T12:14:38.596 INFO:teuthology.orchestra.run.smithi111.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-04T12:14:38.596 INFO:teuthology.orchestra.run.smithi111.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-04T12:14:38.605 INFO:teuthology.orchestra.run.smithi052.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-04T12:14:38.608 INFO:teuthology.orchestra.run.smithi052.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-04T12:14:38.612 INFO:teuthology.orchestra.run.smithi111.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-04T12:14:38.612 INFO:teuthology.orchestra.run.smithi111.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-04T12:14:38.613 INFO:teuthology.orchestra.run.smithi111.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-04T12:14:38.615 INFO:teuthology.orchestra.run.smithi111.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-05-04T12:14:38.616 INFO:teuthology.orchestra.run.smithi111.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-04T12:14:38.617 INFO:teuthology.orchestra.run.smithi052.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-04T12:14:38.617 INFO:teuthology.orchestra.run.smithi111.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-04T12:14:38.620 INFO:teuthology.orchestra.run.smithi052.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-04T12:14:38.621 INFO:teuthology.orchestra.run.smithi052.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-04T12:14:38.629 INFO:teuthology.orchestra.run.smithi052.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-05-04T12:14:38.631 INFO:teuthology.orchestra.run.smithi111.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-05-04T12:14:38.631 INFO:teuthology.orchestra.run.smithi052.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-05-04T12:14:38.633 INFO:teuthology.orchestra.run.smithi111.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-04T12:14:38.633 INFO:teuthology.orchestra.run.smithi052.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-05-04T12:14:38.634 INFO:teuthology.orchestra.run.smithi111.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-04T12:14:38.634 INFO:teuthology.orchestra.run.smithi111.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-04T12:14:38.635 INFO:teuthology.orchestra.run.smithi052.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-05-04T12:14:38.635 INFO:teuthology.orchestra.run.smithi052.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-05-04T12:14:38.647 INFO:teuthology.orchestra.run.smithi052.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-05-04T12:14:38.649 INFO:teuthology.orchestra.run.smithi052.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-05-04T12:14:38.650 INFO:teuthology.orchestra.run.smithi111.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-04T12:14:38.651 INFO:teuthology.orchestra.run.smithi052.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-04T12:14:38.651 INFO:teuthology.orchestra.run.smithi052.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-04T12:14:38.678 INFO:teuthology.orchestra.run.smithi052.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-04T12:14:38.679 INFO:teuthology.orchestra.run.smithi052.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-04T12:14:38.680 INFO:teuthology.orchestra.run.smithi052.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-04T12:14:38.680 INFO:teuthology.orchestra.run.smithi052.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-04T12:14:38.680 INFO:teuthology.orchestra.run.smithi052.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-04T12:14:38.680 INFO:teuthology.orchestra.run.smithi052.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-04T12:14:38.699 INFO:teuthology.orchestra.run.smithi111.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-04T12:14:38.700 INFO:teuthology.orchestra.run.smithi111.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-05-04T12:14:38.701 INFO:teuthology.orchestra.run.smithi111.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-04T12:14:38.701 INFO:teuthology.orchestra.run.smithi111.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-04T12:14:38.702 INFO:teuthology.orchestra.run.smithi111.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-04T12:14:38.705 INFO:teuthology.orchestra.run.smithi052.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-04T12:14:38.706 INFO:teuthology.orchestra.run.smithi052.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-04T12:14:38.709 INFO:teuthology.orchestra.run.smithi052.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-04T12:14:38.709 INFO:teuthology.orchestra.run.smithi052.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-04T12:14:38.720 INFO:teuthology.orchestra.run.smithi111.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-04T12:14:38.722 INFO:teuthology.orchestra.run.smithi111.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-04T12:14:38.722 INFO:teuthology.orchestra.run.smithi111.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-04T12:14:38.734 INFO:teuthology.orchestra.run.smithi052.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-04T12:14:38.735 INFO:teuthology.orchestra.run.smithi052.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-04T12:14:38.736 INFO:teuthology.orchestra.run.smithi052.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-04T12:14:38.736 INFO:teuthology.orchestra.run.smithi052.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-04T12:14:38.736 INFO:teuthology.orchestra.run.smithi052.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-04T12:14:38.738 INFO:teuthology.orchestra.run.smithi052.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-04T12:14:38.742 INFO:teuthology.orchestra.run.smithi111.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-04T12:14:38.742 INFO:teuthology.orchestra.run.smithi111.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-04T12:14:38.749 INFO:teuthology.orchestra.run.smithi111.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-04T12:14:38.749 INFO:teuthology.orchestra.run.smithi111.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-04T12:14:38.750 INFO:teuthology.orchestra.run.smithi111.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-04T12:14:38.750 INFO:teuthology.orchestra.run.smithi111.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-04T12:14:38.751 INFO:teuthology.orchestra.run.smithi111.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-04T12:14:38.751 INFO:teuthology.orchestra.run.smithi111.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-04T12:14:38.754 INFO:teuthology.orchestra.run.smithi111.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-04T12:14:38.755 INFO:teuthology.orchestra.run.smithi052.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-04T12:14:38.755 INFO:teuthology.orchestra.run.smithi052.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-04T12:14:38.761 INFO:teuthology.orchestra.run.smithi052.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-04T12:14:38.762 INFO:teuthology.orchestra.run.smithi052.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-04T12:14:38.768 INFO:teuthology.orchestra.run.smithi111.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-04T12:14:38.770 INFO:teuthology.orchestra.run.smithi111.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-04T12:14:38.770 INFO:teuthology.orchestra.run.smithi111.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-04T12:14:38.774 INFO:teuthology.orchestra.run.smithi052.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-04T12:14:38.775 INFO:teuthology.orchestra.run.smithi052.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-04T12:14:38.776 INFO:teuthology.orchestra.run.smithi052.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-04T12:14:38.779 INFO:teuthology.orchestra.run.smithi052.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-05-04T12:14:38.781 INFO:teuthology.orchestra.run.smithi052.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-04T12:14:38.783 INFO:teuthology.orchestra.run.smithi052.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-04T12:14:38.788 INFO:teuthology.orchestra.run.smithi111.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-04T12:14:38.790 INFO:teuthology.orchestra.run.smithi111.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-05-04T12:14:38.790 INFO:teuthology.orchestra.run.smithi111.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-05-04T12:14:38.791 INFO:teuthology.orchestra.run.smithi111.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-05-04T12:14:38.793 INFO:teuthology.orchestra.run.smithi111.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-04T12:14:38.796 INFO:teuthology.orchestra.run.smithi111.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-05-04T12:14:38.796 INFO:teuthology.orchestra.run.smithi111.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-04T12:14:38.796 INFO:teuthology.orchestra.run.smithi111.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2455-g09dbd6bb-1jammy [3,218 kB] 2024-05-04T12:14:38.798 INFO:teuthology.orchestra.run.smithi052.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-05-04T12:14:38.802 INFO:teuthology.orchestra.run.smithi052.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-04T12:14:38.803 INFO:teuthology.orchestra.run.smithi052.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-04T12:14:38.804 INFO:teuthology.orchestra.run.smithi052.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-04T12:14:38.807 INFO:teuthology.orchestra.run.smithi111.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-04T12:14:38.807 INFO:teuthology.orchestra.run.smithi111.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-04T12:14:38.808 INFO:teuthology.orchestra.run.smithi111.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-04T12:14:38.818 INFO:teuthology.orchestra.run.smithi052.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-04T12:14:38.838 INFO:teuthology.orchestra.run.smithi111.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-04T12:14:38.840 INFO:teuthology.orchestra.run.smithi111.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-04T12:14:38.841 INFO:teuthology.orchestra.run.smithi111.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-04T12:14:38.853 INFO:teuthology.orchestra.run.smithi111.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-04T12:14:38.853 INFO:teuthology.orchestra.run.smithi111.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-04T12:14:38.853 INFO:teuthology.orchestra.run.smithi111.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-04T12:14:38.856 INFO:teuthology.orchestra.run.smithi111.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-04T12:14:38.870 INFO:teuthology.orchestra.run.smithi111.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-04T12:14:38.892 INFO:teuthology.orchestra.run.smithi052.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-04T12:14:38.892 INFO:teuthology.orchestra.run.smithi052.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-05-04T12:14:38.894 INFO:teuthology.orchestra.run.smithi052.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-04T12:14:38.894 INFO:teuthology.orchestra.run.smithi052.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-04T12:14:38.895 INFO:teuthology.orchestra.run.smithi052.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-04T12:14:38.917 INFO:teuthology.orchestra.run.smithi052.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-04T12:14:38.919 INFO:teuthology.orchestra.run.smithi052.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-04T12:14:38.919 INFO:teuthology.orchestra.run.smithi052.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-04T12:14:38.937 INFO:teuthology.orchestra.run.smithi052.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-04T12:14:38.938 INFO:teuthology.orchestra.run.smithi052.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-04T12:14:38.939 INFO:teuthology.orchestra.run.smithi052.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-04T12:14:38.940 INFO:teuthology.orchestra.run.smithi052.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-04T12:14:38.940 INFO:teuthology.orchestra.run.smithi052.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-04T12:14:38.941 INFO:teuthology.orchestra.run.smithi052.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-04T12:14:38.941 INFO:teuthology.orchestra.run.smithi052.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-04T12:14:38.959 INFO:teuthology.orchestra.run.smithi052.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-04T12:14:38.962 INFO:teuthology.orchestra.run.smithi052.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-04T12:14:38.964 INFO:teuthology.orchestra.run.smithi052.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-04T12:14:38.965 INFO:teuthology.orchestra.run.smithi052.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-04T12:14:38.966 INFO:teuthology.orchestra.run.smithi052.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-04T12:14:38.969 INFO:teuthology.orchestra.run.smithi111.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2455-g09dbd6bb-1jammy [711 kB] 2024-05-04T12:14:38.978 INFO:teuthology.orchestra.run.smithi052.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-04T12:14:38.981 INFO:teuthology.orchestra.run.smithi052.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-05-04T12:14:38.982 INFO:teuthology.orchestra.run.smithi052.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-05-04T12:14:38.983 INFO:teuthology.orchestra.run.smithi052.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-05-04T12:14:38.983 INFO:teuthology.orchestra.run.smithi052.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-04T12:14:38.998 INFO:teuthology.orchestra.run.smithi052.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-05-04T12:14:38.998 INFO:teuthology.orchestra.run.smithi052.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-04T12:14:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-04T12:14:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-04T12:14:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-04T12:14:39.005 INFO:teuthology.orchestra.run.smithi111.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2455-g09dbd6bb-1jammy [356 kB] 2024-05-04T12:14:39.013 INFO:teuthology.orchestra.run.smithi111.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2455-g09dbd6bb-1jammy [32.8 kB] 2024-05-04T12:14:39.017 INFO:teuthology.orchestra.run.smithi052.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-04T12:14:39.019 INFO:teuthology.orchestra.run.smithi052.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-04T12:14:39.020 INFO:teuthology.orchestra.run.smithi052.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-04T12:14:39.024 INFO:teuthology.orchestra.run.smithi111.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2455-g09dbd6bb-1jammy [180 kB] 2024-05-04T12:14:39.026 INFO:teuthology.orchestra.run.smithi052.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-04T12:14:39.027 INFO:teuthology.orchestra.run.smithi052.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-04T12:14:39.028 INFO:teuthology.orchestra.run.smithi111.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2455-g09dbd6bb-1jammy [65.7 kB] 2024-05-04T12:14:39.029 INFO:teuthology.orchestra.run.smithi111.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2455-g09dbd6bb-1jammy [330 kB] 2024-05-04T12:14:39.037 INFO:teuthology.orchestra.run.smithi052.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-04T12:14:39.039 INFO:teuthology.orchestra.run.smithi052.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-04T12:14:39.040 INFO:teuthology.orchestra.run.smithi052.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-04T12:14:39.045 INFO:teuthology.orchestra.run.smithi111.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2455-g09dbd6bb-1jammy [6,775 kB] 2024-05-04T12:14:39.311 INFO:teuthology.orchestra.run.smithi111.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2455-g09dbd6bb-1jammy [112 kB] 2024-05-04T12:14:39.313 INFO:teuthology.orchestra.run.smithi111.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2455-g09dbd6bb-1jammy [466 kB] 2024-05-04T12:14:39.332 INFO:teuthology.orchestra.run.smithi111.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2455-g09dbd6bb-1jammy [25.6 MB] 2024-05-04T12:14:39.448 INFO:teuthology.orchestra.run.smithi052.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2455-g09dbd6bb-1jammy [3,218 kB] 2024-05-04T12:14:40.006 INFO:teuthology.orchestra.run.smithi052.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2455-g09dbd6bb-1jammy [711 kB] 2024-05-04T12:14:40.139 INFO:teuthology.orchestra.run.smithi052.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2455-g09dbd6bb-1jammy [356 kB] 2024-05-04T12:14:40.211 INFO:teuthology.orchestra.run.smithi052.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2455-g09dbd6bb-1jammy [32.8 kB] 2024-05-04T12:14:40.214 INFO:teuthology.orchestra.run.smithi052.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2455-g09dbd6bb-1jammy [180 kB] 2024-05-04T12:14:40.240 INFO:teuthology.orchestra.run.smithi052.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2455-g09dbd6bb-1jammy [65.7 kB] 2024-05-04T12:14:40.250 INFO:teuthology.orchestra.run.smithi052.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2455-g09dbd6bb-1jammy [330 kB] 2024-05-04T12:14:40.314 INFO:teuthology.orchestra.run.smithi052.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2455-g09dbd6bb-1jammy [6,775 kB] 2024-05-04T12:14:40.547 INFO:teuthology.orchestra.run.smithi111.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2455-g09dbd6bb-1jammy [5,084 kB] 2024-05-04T12:14:40.626 INFO:teuthology.orchestra.run.smithi111.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2455-g09dbd6bb-1jammy [244 kB] 2024-05-04T12:14:40.629 INFO:teuthology.orchestra.run.smithi111.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2455-g09dbd6bb-1jammy [124 kB] 2024-05-04T12:14:40.630 INFO:teuthology.orchestra.run.smithi111.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2455-g09dbd6bb-1jammy [1,526 kB] 2024-05-04T12:14:40.680 INFO:teuthology.orchestra.run.smithi111.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2455-g09dbd6bb-1jammy [6,246 kB] 2024-05-04T12:14:40.941 INFO:teuthology.orchestra.run.smithi111.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2455-g09dbd6bb-1jammy [22.7 MB] 2024-05-04T12:14:41.300 INFO:teuthology.orchestra.run.smithi052.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2455-g09dbd6bb-1jammy [112 kB] 2024-05-04T12:14:41.305 INFO:teuthology.orchestra.run.smithi052.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2455-g09dbd6bb-1jammy [466 kB] 2024-05-04T12:14:41.379 INFO:teuthology.orchestra.run.smithi052.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2455-g09dbd6bb-1jammy [25.6 MB] 2024-05-04T12:14:41.975 INFO:teuthology.orchestra.run.smithi111.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2455-g09dbd6bb-1jammy [14.0 kB] 2024-05-04T12:14:41.975 INFO:teuthology.orchestra.run.smithi111.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2455-g09dbd6bb-1jammy [869 kB] 2024-05-04T12:14:41.992 INFO:teuthology.orchestra.run.smithi111.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2455-g09dbd6bb-1jammy [2,437 kB] 2024-05-04T12:14:42.033 INFO:teuthology.orchestra.run.smithi111.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2455-g09dbd6bb-1jammy [790 kB] 2024-05-04T12:14:42.061 INFO:teuthology.orchestra.run.smithi111.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2455-g09dbd6bb-1jammy [147 kB] 2024-05-04T12:14:42.065 INFO:teuthology.orchestra.run.smithi111.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2455-g09dbd6bb-1jammy [2,293 kB] 2024-05-04T12:14:42.161 INFO:teuthology.orchestra.run.smithi111.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2455-g09dbd6bb-1jammy [8,625 kB] 2024-05-04T12:14:42.520 INFO:teuthology.orchestra.run.smithi111.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2455-g09dbd6bb-1jammy [14.2 kB] 2024-05-04T12:14:42.521 INFO:teuthology.orchestra.run.smithi111.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2455-g09dbd6bb-1jammy [50.7 MB] 2024-05-04T12:14:44.879 INFO:teuthology.orchestra.run.smithi111.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2455-g09dbd6bb-1jammy [121 kB] 2024-05-04T12:14:44.881 INFO:teuthology.orchestra.run.smithi111.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2455-g09dbd6bb-1jammy [229 kB] 2024-05-04T12:14:44.886 INFO:teuthology.orchestra.run.smithi111.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2455-g09dbd6bb-1jammy [34.0 kB] 2024-05-04T12:14:44.887 INFO:teuthology.orchestra.run.smithi111.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2455-g09dbd6bb-1jammy [23.4 kB] 2024-05-04T12:14:44.888 INFO:teuthology.orchestra.run.smithi111.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2455-g09dbd6bb-1jammy [38.9 kB] 2024-05-04T12:14:44.889 INFO:teuthology.orchestra.run.smithi111.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2455-g09dbd6bb-1jammy [13.4 MB] 2024-05-04T12:14:45.240 INFO:teuthology.orchestra.run.smithi052.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2455-g09dbd6bb-1jammy [5,084 kB] 2024-05-04T12:14:45.299 INFO:teuthology.orchestra.run.smithi111.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2455-g09dbd6bb-1jammy [91.2 kB] 2024-05-04T12:14:45.574 INFO:teuthology.orchestra.run.smithi052.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2455-g09dbd6bb-1jammy [244 kB] 2024-05-04T12:14:45.599 INFO:teuthology.orchestra.run.smithi052.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2455-g09dbd6bb-1jammy [124 kB] 2024-05-04T12:14:45.602 INFO:teuthology.orchestra.run.smithi052.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2455-g09dbd6bb-1jammy [1,526 kB] 2024-05-04T12:14:45.706 INFO:teuthology.orchestra.run.smithi052.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2455-g09dbd6bb-1jammy [6,246 kB] 2024-05-04T12:14:45.914 INFO:teuthology.orchestra.run.smithi111.stdout:Fetched 180 MB in 7s (25.5 MB/s) 2024-05-04T12:14:46.043 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-04T12:14:46.106 INFO:teuthology.orchestra.run.smithi052.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2455-g09dbd6bb-1jammy [22.7 MB] 2024-05-04T12:14:46.136 INFO:teuthology.orchestra.run.smithi111.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-05-04T12:14:46.141 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-05-04T12:14:46.183 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-05-04T12:14:46.478 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-04T12:14:46.504 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-04T12:14:46.536 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-04T12:14:46.839 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-cmd2. 2024-05-04T12:14:46.865 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-05-04T12:14:46.905 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-05-04T12:14:47.250 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-04T12:14:47.276 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-04T12:14:47.383 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-04T12:14:47.667 INFO:teuthology.orchestra.run.smithi052.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2455-g09dbd6bb-1jammy [14.0 kB] 2024-05-04T12:14:47.667 INFO:teuthology.orchestra.run.smithi052.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2455-g09dbd6bb-1jammy [869 kB] 2024-05-04T12:14:47.685 INFO:teuthology.orchestra.run.smithi052.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2455-g09dbd6bb-1jammy [2,437 kB] 2024-05-04T12:14:47.737 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-04T12:14:47.763 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-04T12:14:47.803 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-04T12:14:47.823 INFO:teuthology.orchestra.run.smithi052.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2455-g09dbd6bb-1jammy [790 kB] 2024-05-04T12:14:47.888 INFO:teuthology.orchestra.run.smithi052.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2455-g09dbd6bb-1jammy [147 kB] 2024-05-04T12:14:47.891 INFO:teuthology.orchestra.run.smithi052.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2455-g09dbd6bb-1jammy [2,293 kB] 2024-05-04T12:14:48.038 INFO:teuthology.orchestra.run.smithi052.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2455-g09dbd6bb-1jammy [8,625 kB] 2024-05-04T12:14:48.148 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-04T12:14:48.174 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-04T12:14:48.214 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-04T12:14:48.576 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-04T12:14:48.599 INFO:teuthology.orchestra.run.smithi052.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2455-g09dbd6bb-1jammy [14.2 kB] 2024-05-04T12:14:48.600 INFO:teuthology.orchestra.run.smithi052.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2455-g09dbd6bb-1jammy [50.7 MB] 2024-05-04T12:14:48.602 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-04T12:14:48.651 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-04T12:14:49.063 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-04T12:14:49.089 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-04T12:14:49.129 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-04T12:14:49.457 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-04T12:14:49.483 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-04T12:14:49.524 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-04T12:14:49.837 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-04T12:14:49.861 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-04T12:14:49.910 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-04T12:14:50.230 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package librados2. 2024-05-04T12:14:50.256 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../010-librados2_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:14:50.296 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking librados2 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:14:50.724 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package librbd1. 2024-05-04T12:14:50.750 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../011-librbd1_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:14:50.790 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking librbd1 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:14:51.218 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libcephfs2. 2024-05-04T12:14:51.244 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:14:51.285 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libcephfs2 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:14:51.562 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-rados. 2024-05-04T12:14:51.577 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:14:51.620 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-rados (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:14:51.949 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-04T12:14:51.975 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:14:52.015 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-ceph-argparse (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:14:52.275 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-cephfs. 2024-05-04T12:14:52.291 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:14:52.334 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-cephfs (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:14:52.604 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-04T12:14:52.630 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:14:52.662 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-ceph-common (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:14:52.745 INFO:teuthology.orchestra.run.smithi052.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2455-g09dbd6bb-1jammy [121 kB] 2024-05-04T12:14:52.748 INFO:teuthology.orchestra.run.smithi052.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2455-g09dbd6bb-1jammy [229 kB] 2024-05-04T12:14:52.752 INFO:teuthology.orchestra.run.smithi052.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2455-g09dbd6bb-1jammy [34.0 kB] 2024-05-04T12:14:52.753 INFO:teuthology.orchestra.run.smithi052.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2455-g09dbd6bb-1jammy [23.4 kB] 2024-05-04T12:14:52.754 INFO:teuthology.orchestra.run.smithi052.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2455-g09dbd6bb-1jammy [38.9 kB] 2024-05-04T12:14:52.755 INFO:teuthology.orchestra.run.smithi052.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2455-g09dbd6bb-1jammy [13.4 MB] 2024-05-04T12:14:52.931 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-prettytable. 2024-05-04T12:14:52.946 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-05-04T12:14:52.989 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-04T12:14:53.250 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-rbd. 2024-05-04T12:14:53.277 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:14:53.317 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-rbd (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:14:53.670 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-04T12:14:53.697 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-04T12:14:53.737 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-04T12:14:54.014 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-05-04T12:14:54.030 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-05-04T12:14:54.058 INFO:teuthology.orchestra.run.smithi052.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-29-0642/09dbd6bba52d19ba471c33bc15c008e6ad158ea6/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2455-g09dbd6bb-1jammy [91.2 kB] 2024-05-04T12:14:54.064 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-05-04T12:14:54.325 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-05-04T12:14:54.340 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-05-04T12:14:54.384 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-04T12:14:54.719 INFO:teuthology.orchestra.run.smithi052.stdout:Fetched 180 MB in 16s (11.5 MB/s) 2024-05-04T12:14:54.720 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package lua5.1. 2024-05-04T12:14:54.747 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-05-04T12:14:54.787 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-05-04T12:14:54.852 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-04T12:14:54.954 INFO:teuthology.orchestra.run.smithi052.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-05-04T12:14:54.960 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-05-04T12:14:54.992 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-05-04T12:14:55.156 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package lua-any. 2024-05-04T12:14:55.183 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-05-04T12:14:55.223 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking lua-any (27ubuntu1) ... 2024-05-04T12:14:55.236 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-04T12:14:55.252 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-04T12:14:55.286 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-04T12:14:55.434 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package zip. 2024-05-04T12:14:55.449 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-05-04T12:14:55.483 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking zip (3.0-12build2) ... 2024-05-04T12:14:55.597 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-cmd2. 2024-05-04T12:14:55.623 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-05-04T12:14:55.663 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-05-04T12:14:55.795 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package unzip. 2024-05-04T12:14:55.822 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-05-04T12:14:55.862 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-05-04T12:14:56.050 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-04T12:14:56.076 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-04T12:14:56.158 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-04T12:14:56.257 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package luarocks. 2024-05-04T12:14:56.284 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-05-04T12:14:56.324 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-05-04T12:14:56.503 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-04T12:14:56.530 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-04T12:14:56.570 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-04T12:14:56.681 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package librgw2. 2024-05-04T12:14:56.708 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../027-librgw2_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:14:56.748 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking librgw2 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:14:56.906 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-04T12:14:56.933 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-04T12:14:56.964 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-04T12:14:57.267 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-rgw. 2024-05-04T12:14:57.282 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:14:57.325 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-rgw (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:14:57.401 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-04T12:14:57.428 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-04T12:14:57.476 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-04T12:14:57.670 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-04T12:14:57.697 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-04T12:14:57.736 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-04T12:14:57.887 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-04T12:14:57.914 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-04T12:14:57.954 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-04T12:14:58.056 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libradosstriper1. 2024-05-04T12:14:58.083 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:14:58.123 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libradosstriper1 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:14:58.274 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-04T12:14:58.300 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-04T12:14:58.331 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-04T12:14:58.476 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph-common. 2024-05-04T12:14:58.502 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:14:58.534 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph-common (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:14:58.635 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-04T12:14:58.661 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-04T12:14:58.701 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-04T12:14:59.021 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package librados2. 2024-05-04T12:14:59.048 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../010-librados2_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:14:59.087 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking librados2 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:14:59.723 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package librbd1. 2024-05-04T12:14:59.750 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../011-librbd1_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:14:59.790 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking librbd1 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:14:59.973 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph-base. 2024-05-04T12:14:59.999 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:00.048 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph-base (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:00.310 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libcephfs2. 2024-05-04T12:15:00.336 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:00.367 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libcephfs2 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:00.618 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-04T12:15:00.620 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-rados. 2024-05-04T12:15:00.636 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:00.645 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-04T12:15:00.670 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-rados (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:00.685 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-04T12:15:00.964 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-04T12:15:00.988 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-cheroot. 2024-05-04T12:15:00.991 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:01.015 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-04T12:15:01.030 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-ceph-argparse (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:01.054 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-04T12:15:01.292 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-cephfs. 2024-05-04T12:15:01.318 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:01.349 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-04T12:15:01.375 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-cephfs (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:01.376 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-04T12:15:01.407 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-04T12:15:01.686 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-04T12:15:01.693 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-04T12:15:01.713 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:01.720 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-04T12:15:01.752 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-ceph-common (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:01.760 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-04T12:15:02.055 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-prettytable. 2024-05-04T12:15:02.063 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-04T12:15:02.071 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-05-04T12:15:02.090 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-04T12:15:02.105 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-04T12:15:02.130 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-04T12:15:02.324 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-rbd. 2024-05-04T12:15:02.340 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:02.382 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-rbd (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:02.424 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-tempora. 2024-05-04T12:15:02.450 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-05-04T12:15:02.490 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-04T12:15:02.727 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-04T12:15:02.754 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-04T12:15:02.768 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-portend. 2024-05-04T12:15:02.794 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-04T12:15:02.795 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-05-04T12:15:02.826 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-04T12:15:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-05-04T12:15:03.098 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-05-04T12:15:03.129 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-04T12:15:03.137 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-05-04T12:15:03.156 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-04T12:15:03.196 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-04T12:15:03.441 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-05-04T12:15:03.456 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-05-04T12:15:03.491 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-04T12:15:03.499 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-04T12:15:03.525 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-04T12:15:03.565 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-04T12:15:03.827 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package lua5.1. 2024-05-04T12:15:03.854 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-05-04T12:15:03.893 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-05-04T12:15:03.902 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-natsort. 2024-05-04T12:15:03.917 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-05-04T12:15:03.952 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-04T12:15:04.205 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package lua-any. 2024-05-04T12:15:04.221 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-05-04T12:15:04.239 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-logutils. 2024-05-04T12:15:04.263 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking lua-any (27ubuntu1) ... 2024-05-04T12:15:04.265 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-05-04T12:15:04.305 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-04T12:15:04.507 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package zip. 2024-05-04T12:15:04.534 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-05-04T12:15:04.573 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking zip (3.0-12build2) ... 2024-05-04T12:15:04.608 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-04T12:15:04.634 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-04T12:15:04.674 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-04T12:15:04.860 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package unzip. 2024-05-04T12:15:04.887 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-05-04T12:15:04.918 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-05-04T12:15:04.960 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-mako. 2024-05-04T12:15:04.987 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-04T12:15:05.027 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-04T12:15:05.330 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-04T12:15:05.331 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package luarocks. 2024-05-04T12:15:05.357 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-04T12:15:05.357 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-05-04T12:15:05.397 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-04T12:15:05.397 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-05-04T12:15:05.641 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-04T12:15:05.656 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-04T12:15:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-04T12:15:05.779 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package librgw2. 2024-05-04T12:15:05.807 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../027-librgw2_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:05.845 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking librgw2 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:06.002 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-webob. 2024-05-04T12:15:06.028 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-04T12:15:06.068 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-04T12:15:06.406 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-rgw. 2024-05-04T12:15:06.422 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:06.430 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-waitress. 2024-05-04T12:15:06.456 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-rgw (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:06.456 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-04T12:15:06.499 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-04T12:15:06.753 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-04T12:15:06.780 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-04T12:15:06.819 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-04T12:15:06.916 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-tempita. 2024-05-04T12:15:06.943 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-04T12:15:06.983 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-04T12:15:07.131 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libradosstriper1. 2024-05-04T12:15:07.157 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:07.197 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libradosstriper1 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:07.244 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-paste. 2024-05-04T12:15:07.270 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-04T12:15:07.302 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-04T12:15:07.584 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-common. 2024-05-04T12:15:07.611 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:07.630 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-04T12:15:07.642 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-common (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:07.657 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-04T12:15:07.697 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-04T12:15:07.958 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-04T12:15:07.984 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-04T12:15:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-04T12:15:08.286 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-webtest. 2024-05-04T12:15:08.312 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-05-04T12:15:08.352 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-04T12:15:08.666 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-pecan. 2024-05-04T12:15:08.692 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-04T12:15:08.732 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-04T12:15:09.095 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-certifi. 2024-05-04T12:15:09.122 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-05-04T12:15:09.162 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-04T12:15:09.164 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-base. 2024-05-04T12:15:09.191 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:09.240 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-base (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:09.465 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-idna. 2024-05-04T12:15:09.480 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-05-04T12:15:09.515 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-idna (3.3-1) ... 2024-05-04T12:15:09.810 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-urllib3. 2024-05-04T12:15:09.836 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-04T12:15:09.876 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-04T12:15:10.077 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-04T12:15:10.093 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-04T12:15:10.135 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-04T12:15:10.163 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-requests. 2024-05-04T12:15:10.189 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-04T12:15:10.229 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-04T12:15:10.396 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-cheroot. 2024-05-04T12:15:10.423 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-04T12:15:10.463 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-04T12:15:10.566 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-04T12:15:10.592 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-05-04T12:15:10.632 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-04T12:15:10.783 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-04T12:15:10.810 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-04T12:15:10.849 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-04T12:15:10.902 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-04T12:15:10.929 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:10.968 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:11.135 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-04T12:15:11.162 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-04T12:15:11.218 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-04T12:15:11.366 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-04T12:15:11.392 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:11.732 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:12.239 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-04T12:15:12.267 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-04T12:15:12.297 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-04T12:15:12.558 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-tempora. 2024-05-04T12:15:12.585 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-05-04T12:15:12.624 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-04T12:15:12.860 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-04T12:15:12.887 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-04T12:15:12.918 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-04T12:15:12.927 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-portend. 2024-05-04T12:15:12.954 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-05-04T12:15:13.002 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-04T12:15:13.188 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-openssl. 2024-05-04T12:15:13.214 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-05-04T12:15:13.246 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-04T12:15:13.280 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-04T12:15:13.296 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-04T12:15:13.330 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-04T12:15:13.558 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph-mgr. 2024-05-04T12:15:13.558 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-04T12:15:13.584 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:13.585 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-04T12:15:13.616 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-04T12:15:13.624 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph-mgr (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:13.970 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-natsort. 2024-05-04T12:15:13.997 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-05-04T12:15:14.027 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph-mon. 2024-05-04T12:15:14.036 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-04T12:15:14.054 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:14.094 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph-mon (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:14.331 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-logutils. 2024-05-04T12:15:14.358 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-05-04T12:15:14.398 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-04T12:15:14.681 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph-osd. 2024-05-04T12:15:14.701 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-04T12:15:14.708 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:14.727 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-04T12:15:14.739 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph-osd (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:14.767 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-04T12:15:15.029 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-mako. 2024-05-04T12:15:15.055 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-04T12:15:15.103 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-04T12:15:15.423 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-04T12:15:15.450 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-04T12:15:15.490 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-04T12:15:15.776 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph. 2024-05-04T12:15:15.792 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-04T12:15:15.803 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../068-ceph_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:15.820 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-04T12:15:15.842 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:15.859 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-04T12:15:16.146 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-webob. 2024-05-04T12:15:16.161 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph-fuse. 2024-05-04T12:15:16.172 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-04T12:15:16.188 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:16.212 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-04T12:15:16.228 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph-fuse (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:16.506 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-waitress. 2024-05-04T12:15:16.523 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-04T12:15:16.560 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-04T12:15:16.573 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph-mds. 2024-05-04T12:15:16.600 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:16.639 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph-mds (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:16.860 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-tempita. 2024-05-04T12:15:16.886 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-04T12:15:16.926 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-04T12:15:16.975 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package cephadm. 2024-05-04T12:15:16.991 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../071-cephadm_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:17.034 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking cephadm (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:17.204 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-paste. 2024-05-04T12:15:17.231 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-04T12:15:17.270 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-04T12:15:17.354 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-04T12:15:17.381 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-04T12:15:17.420 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-04T12:15:17.607 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-04T12:15:17.623 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-04T12:15:17.665 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-04T12:15:17.723 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-04T12:15:17.738 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-04T12:15:17.781 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-04T12:15:17.910 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-04T12:15:17.926 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-04T12:15:17.959 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-04T12:15:18.238 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-webtest. 2024-05-04T12:15:18.265 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-05-04T12:15:18.296 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-04T12:15:18.532 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pecan. 2024-05-04T12:15:18.548 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-04T12:15:18.582 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-04T12:15:18.845 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-babel. 2024-05-04T12:15:18.872 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-04T12:15:18.911 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-04T12:15:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-certifi. 2024-05-04T12:15:18.938 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-05-04T12:15:18.978 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-04T12:15:19.239 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-jinja2. 2024-05-04T12:15:19.265 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-05-04T12:15:19.298 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-idna. 2024-05-04T12:15:19.314 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-04T12:15:19.324 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-05-04T12:15:19.364 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-idna (3.3-1) ... 2024-05-04T12:15:19.675 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-04T12:15:19.676 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-urllib3. 2024-05-04T12:15:19.702 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:19.703 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-04T12:15:19.742 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:19.742 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-04T12:15:20.079 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-04T12:15:20.079 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-requests. 2024-05-04T12:15:20.105 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-05-04T12:15:20.106 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-04T12:15:20.145 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-04T12:15:20.147 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-04T12:15:20.448 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-routes. 2024-05-04T12:15:20.475 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-04T12:15:20.514 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-04T12:15:20.565 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-04T12:15:20.581 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-05-04T12:15:20.615 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-04T12:15:20.851 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-04T12:15:20.878 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:20.884 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-04T12:15:20.900 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:20.917 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:20.934 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:21.331 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-04T12:15:21.358 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:21.397 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:21.709 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-04T12:15:21.736 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-04T12:15:21.775 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-04T12:15:22.011 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-openssl. 2024-05-04T12:15:22.038 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-05-04T12:15:22.069 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-04T12:15:22.239 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-04T12:15:22.255 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-04T12:15:22.289 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-04T12:15:22.423 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mgr. 2024-05-04T12:15:22.449 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:22.489 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mgr (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:22.751 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-joblib. 2024-05-04T12:15:22.775 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-04T12:15:22.809 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-04T12:15:22.909 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mon. 2024-05-04T12:15:22.936 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:22.975 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mon (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:23.130 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-04T12:15:23.157 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-04T12:15:23.273 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-04T12:15:23.521 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-osd. 2024-05-04T12:15:23.548 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:23.579 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-osd (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:23.593 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-sklearn. 2024-05-04T12:15:23.620 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-04T12:15:23.659 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-04T12:15:24.377 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-04T12:15:24.405 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:24.443 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:24.749 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph. 2024-05-04T12:15:24.765 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../068-ceph_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:24.807 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:25.126 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-fuse. 2024-05-04T12:15:25.153 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:25.192 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-fuse (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:25.397 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-cachetools. 2024-05-04T12:15:25.425 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-05-04T12:15:25.455 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-04T12:15:25.554 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mds. 2024-05-04T12:15:25.581 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:25.614 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mds (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:25.758 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-04T12:15:25.785 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-04T12:15:25.825 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-04T12:15:25.990 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package cephadm. 2024-05-04T12:15:26.017 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../071-cephadm_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:26.057 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking cephadm (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:26.153 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-04T12:15:26.181 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-04T12:15:26.220 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-04T12:15:26.376 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-04T12:15:26.404 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-04T12:15:26.442 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-04T12:15:26.531 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-rsa. 2024-05-04T12:15:26.559 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-05-04T12:15:26.606 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-04T12:15:26.812 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-04T12:15:26.828 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-04T12:15:26.879 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-04T12:15:26.918 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-google-auth. 2024-05-04T12:15:26.946 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-05-04T12:15:26.985 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-04T12:15:27.271 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-04T12:15:27.299 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-04T12:15:27.329 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-04T12:15:27.633 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-websocket. 2024-05-04T12:15:27.660 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-05-04T12:15:27.699 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-04T12:15:27.942 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-babel. 2024-05-04T12:15:27.970 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-04T12:15:28.008 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-04T12:15:28.020 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-04T12:15:28.048 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-04T12:15:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-04T12:15:28.320 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-jinja2. 2024-05-04T12:15:28.348 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-05-04T12:15:28.386 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-04T12:15:28.731 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-04T12:15:28.759 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:28.798 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:28.942 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-04T12:15:28.958 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:28.992 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:29.135 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-04T12:15:29.162 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-05-04T12:15:29.201 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-04T12:15:29.278 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-04T12:15:29.305 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-04T12:15:29.344 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-04T12:15:29.470 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-routes. 2024-05-04T12:15:29.498 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-04T12:15:29.528 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-04T12:15:29.706 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-04T12:15:29.734 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-04T12:15:29.772 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-04T12:15:29.824 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-04T12:15:29.851 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:29.881 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:30.034 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package jq. 2024-05-04T12:15:30.050 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-04T12:15:30.083 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-04T12:15:30.278 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package socat. 2024-05-04T12:15:30.294 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-04T12:15:30.327 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-04T12:15:30.614 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package xmlstarlet. 2024-05-04T12:15:30.630 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-04T12:15:30.672 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-04T12:15:31.002 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph-test. 2024-05-04T12:15:31.018 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:31.051 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph-test (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:31.270 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-04T12:15:31.287 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-04T12:15:31.328 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-04T12:15:31.807 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-joblib. 2024-05-04T12:15:31.835 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-04T12:15:31.873 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-04T12:15:32.269 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-04T12:15:32.297 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-04T12:15:32.421 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-04T12:15:32.707 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-sklearn. 2024-05-04T12:15:32.735 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-04T12:15:32.773 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-04T12:15:33.398 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ceph-volume. 2024-05-04T12:15:33.425 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:33.480 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ceph-volume (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:33.483 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-04T12:15:33.511 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:33.549 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:33.860 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-04T12:15:33.888 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:33.926 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking cephfs-mirror (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:34.246 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-colorama. 2024-05-04T12:15:34.274 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-05-04T12:15:34.312 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-05-04T12:15:34.582 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package cephfs-shell. 2024-05-04T12:15:34.586 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-cachetools. 2024-05-04T12:15:34.609 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:34.614 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-05-04T12:15:34.648 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking cephfs-shell (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:34.652 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-04T12:15:34.876 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package cephfs-top. 2024-05-04T12:15:34.892 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:34.926 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking cephfs-top (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:34.930 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-04T12:15:34.947 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-04T12:15:34.988 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-04T12:15:35.179 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-04T12:15:35.206 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:35.245 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libcephfs-dev (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:35.283 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-04T12:15:35.300 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-04T12:15:35.333 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-04T12:15:35.573 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package lua-socket:amd64. 2024-05-04T12:15:35.600 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-05-04T12:15:35.611 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-rsa. 2024-05-04T12:15:35.639 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-05-04T12:15:35.639 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-04T12:15:35.677 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-04T12:15:35.978 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package lua-sec:amd64. 2024-05-04T12:15:35.998 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-google-auth. 2024-05-04T12:15:36.005 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-05-04T12:15:36.026 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-05-04T12:15:36.044 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-05-04T12:15:36.090 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-04T12:15:36.398 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package nvme-cli. 2024-05-04T12:15:36.425 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-04T12:15:36.426 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-04T12:15:36.454 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-04T12:15:36.464 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-04T12:15:36.493 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-04T12:15:36.779 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-websocket. 2024-05-04T12:15:36.807 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-05-04T12:15:36.846 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-04T12:15:36.861 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package pkg-config. 2024-05-04T12:15:36.889 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-05-04T12:15:36.928 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-05-04T12:15:37.116 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-04T12:15:37.144 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-04T12:15:37.182 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-04T12:15:37.189 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-04T12:15:37.217 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-04T12:15:37.255 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-04T12:15:37.659 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-04T12:15:37.675 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-04T12:15:37.717 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-04T12:15:37.880 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-04T12:15:37.897 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:37.946 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:38.020 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-pastescript. 2024-05-04T12:15:38.047 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-05-04T12:15:38.086 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-04T12:15:38.274 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-04T12:15:38.302 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-04T12:15:38.341 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-04T12:15:38.406 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-pluggy. 2024-05-04T12:15:38.434 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-04T12:15:38.473 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-04T12:15:38.685 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-04T12:15:38.713 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-04T12:15:38.752 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-04T12:15:38.768 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-psutil. 2024-05-04T12:15:38.795 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-04T12:15:38.834 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-04T12:15:39.021 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package jq. 2024-05-04T12:15:39.046 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-04T12:15:39.087 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-04T12:15:39.196 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-py. 2024-05-04T12:15:39.223 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-05-04T12:15:39.270 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-04T12:15:39.315 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package socat. 2024-05-04T12:15:39.332 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-04T12:15:39.365 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-04T12:15:39.599 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-pygments. 2024-05-04T12:15:39.626 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-04T12:15:39.668 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package xmlstarlet. 2024-05-04T12:15:39.673 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-04T12:15:39.685 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-04T12:15:39.727 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-04T12:15:40.065 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-test. 2024-05-04T12:15:40.078 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-04T12:15:40.081 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:40.105 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-04T12:15:40.122 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-test (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:40.144 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-04T12:15:40.464 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-toml. 2024-05-04T12:15:40.491 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-05-04T12:15:40.530 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-04T12:15:40.825 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-pytest. 2024-05-04T12:15:40.852 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-04T12:15:40.891 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-04T12:15:41.278 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package python3-simplejson. 2024-05-04T12:15:41.305 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-04T12:15:41.344 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-04T12:15:41.614 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-04T12:15:41.630 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-04T12:15:41.672 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-04T12:15:42.410 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package radosgw. 2024-05-04T12:15:42.438 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../122-radosgw_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:42.476 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking radosgw (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:42.477 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-volume. 2024-05-04T12:15:42.505 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:42.543 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-volume (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:42.923 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-04T12:15:42.951 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:42.989 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking cephfs-mirror (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:43.254 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package rbd-fuse. 2024-05-04T12:15:43.270 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:43.304 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking rbd-fuse (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:43.309 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-colorama. 2024-05-04T12:15:43.337 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-05-04T12:15:43.375 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-05-04T12:15:43.600 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-04T12:15:43.628 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package cephfs-shell. 2024-05-04T12:15:43.644 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:43.677 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking cephfs-shell (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:43.931 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package cephfs-top. 2024-05-04T12:15:43.958 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2455-g09dbd6bb-1jammy_all.deb ... 2024-05-04T12:15:43.997 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking cephfs-top (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:44.001 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-04T12:15:44.102 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-04T12:15:44.250 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-04T12:15:44.278 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:44.316 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libcephfs-dev (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:44.347 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-04T12:15:44.611 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package lua-socket:amd64. 2024-05-04T12:15:44.639 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-05-04T12:15:44.668 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-04T12:15:44.694 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-04T12:15:44.982 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package lua-sec:amd64. 2024-05-04T12:15:45.010 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-05-04T12:15:45.048 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-05-04T12:15:45.122 INFO:teuthology.orchestra.run.smithi111.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-04T12:15:45.195 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up cephadm (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:45.410 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package nvme-cli. 2024-05-04T12:15:45.439 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-04T12:15:45.485 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-04T12:15:45.679 INFO:teuthology.orchestra.run.smithi111.stdout:Adding system user cephadm....done 2024-05-04T12:15:45.835 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-04T12:15:45.909 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package pkg-config. 2024-05-04T12:15:45.938 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-05-04T12:15:45.975 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-05-04T12:15:46.112 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-04T12:15:46.195 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-04T12:15:46.223 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-04T12:15:46.253 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-04T12:15:46.346 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-04T12:15:46.472 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-04T12:15:46.648 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-04T12:15:46.676 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-04T12:15:46.706 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-04T12:15:46.723 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-04T12:15:46.992 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-04T12:15:46.992 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pastescript. 2024-05-04T12:15:47.021 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-05-04T12:15:47.059 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-04T12:15:47.117 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-04T12:15:47.379 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pluggy. 2024-05-04T12:15:47.407 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-04T12:15:47.427 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-04T12:15:47.445 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-04T12:15:47.757 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-psutil. 2024-05-04T12:15:47.771 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-04T12:15:47.785 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-04T12:15:47.823 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-04T12:15:48.030 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-05-04T12:15:48.176 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-py. 2024-05-04T12:15:48.205 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-05-04T12:15:48.242 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-04T12:15:48.299 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-04T12:15:48.571 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pygments. 2024-05-04T12:15:48.575 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-05-04T12:15:48.600 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-04T12:15:48.637 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-04T12:15:48.835 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-04T12:15:49.113 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-04T12:15:49.125 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-04T12:15:49.141 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-04T12:15:49.183 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-04T12:15:49.362 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-04T12:15:49.494 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-toml. 2024-05-04T12:15:49.522 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-05-04T12:15:49.561 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-04T12:15:49.688 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-ceph-argparse (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:49.830 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pytest. 2024-05-04T12:15:49.847 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-04T12:15:49.880 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-04T12:15:49.947 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-04T12:15:50.076 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-04T12:15:50.200 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-simplejson. 2024-05-04T12:15:50.219 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-05-04T12:15:50.229 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-04T12:15:50.266 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-04T12:15:50.470 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-05-04T12:15:50.529 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-04T12:15:50.544 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-04T12:15:50.571 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-04T12:15:50.586 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-04T12:15:50.813 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-05-04T12:15:51.176 INFO:teuthology.orchestra.run.smithi111.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-05-04T12:15:51.203 INFO:teuthology.orchestra.run.smithi111.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-05-04T12:15:51.266 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-04T12:15:51.315 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package radosgw. 2024-05-04T12:15:51.343 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../122-radosgw_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:51.381 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking radosgw (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:51.393 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-04T12:15:51.744 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-04T12:15:51.853 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-04T12:15:52.113 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-04T12:15:52.260 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package rbd-fuse. 2024-05-04T12:15:52.276 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2455-g09dbd6bb-1jammy_amd64.deb ... 2024-05-04T12:15:52.317 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking rbd-fuse (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:52.364 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-04T12:15:52.623 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-04T12:15:52.707 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up zip (3.0-12build2) ... 2024-05-04T12:15:52.833 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-04T12:15:52.873 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-04T12:15:52.990 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-04T12:15:53.266 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-04T12:15:53.527 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-04T12:15:53.581 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-04T12:15:53.770 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-04T12:15:53.871 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-04T12:15:53.972 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-04T12:15:53.977 INFO:teuthology.orchestra.run.smithi052.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-04T12:15:54.042 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up cephadm (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:54.265 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-04T12:15:54.567 INFO:teuthology.orchestra.run.smithi052.stdout:Adding system user cephadm....done 2024-05-04T12:15:54.649 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-04T12:15:54.715 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-04T12:15:54.909 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-04T12:15:55.058 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-04T12:15:55.286 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-04T12:15:55.317 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-04T12:15:55.443 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-04T12:15:55.562 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-05-04T12:15:55.703 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-04T12:15:55.863 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-04T12:15:55.970 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-04T12:15:56.088 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-04T12:15:56.206 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-04T12:15:56.398 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-04T12:15:56.480 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-04T12:15:56.608 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-ceph-common (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:56.742 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-04T12:15:56.910 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-idna (3.3-1) ... 2024-05-04T12:15:57.001 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-05-04T12:15:57.211 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-04T12:15:57.261 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-04T12:15:57.562 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-05-04T12:15:57.814 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-04T12:15:58.082 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-04T12:15:58.341 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-04T12:15:58.342 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-05-04T12:15:58.594 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-04T12:15:58.601 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-ceph-argparse (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:15:58.711 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-04T12:15:58.877 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-04T12:15:58.962 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-04T12:15:59.003 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-04T12:15:59.088 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-04T12:15:59.129 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-05-04T12:15:59.214 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-04T12:15:59.396 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-05-04T12:15:59.523 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-04T12:15:59.607 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-04T12:15:59.807 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-05-04T12:15:59.867 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-04T12:16:00.118 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-04T12:16:00.219 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-04T12:16:00.268 INFO:teuthology.orchestra.run.smithi052.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-05-04T12:16:00.298 INFO:teuthology.orchestra.run.smithi052.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-05-04T12:16:00.360 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-04T12:16:00.486 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-04T12:16:00.504 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-04T12:16:00.755 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-04T12:16:00.854 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-04T12:16:00.980 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-04T12:16:01.014 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-04T12:16:01.248 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-04T12:16:01.274 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-04T12:16:01.507 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-04T12:16:01.533 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-04T12:16:01.801 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-04T12:16:01.817 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up zip (3.0-12build2) ... 2024-05-04T12:16:01.911 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-04T12:16:01.943 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-04T12:16:02.170 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-04T12:16:02.271 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-04T12:16:02.564 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-04T12:16:02.637 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-04T12:16:02.865 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-04T12:16:02.905 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-04T12:16:03.030 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-04T12:16:03.133 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-04T12:16:03.140 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-04T12:16:03.259 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-05-04T12:16:03.377 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-04T12:16:03.432 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-04T12:16:03.450 INFO:teuthology.orchestra.run.smithi111.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-04T12:16:03.720 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-04T12:16:03.817 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-04T12:16:03.846 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-04T12:16:04.069 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-04T12:16:04.256 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-04T12:16:04.453 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-04T12:16:04.516 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up lua-any (27ubuntu1) ... 2024-05-04T12:16:04.642 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-04T12:16:04.755 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-05-04T12:16:04.901 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-04T12:16:05.011 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-04T12:16:05.081 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-04T12:16:05.295 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-04T12:16:05.441 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-04T12:16:05.597 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-04T12:16:05.692 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-04T12:16:05.818 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-ceph-common (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:05.889 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-04T12:16:06.101 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-04T12:16:06.128 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-idna (3.3-1) ... 2024-05-04T12:16:06.353 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-04T12:16:06.445 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-04T12:16:06.612 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-04T12:16:06.863 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-04T12:16:07.249 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-04T12:16:07.541 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-04T12:16:07.660 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-05-04T12:16:07.911 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-04T12:16:07.961 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-04T12:16:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up librados2 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:08.096 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-04T12:16:08.138 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:08.255 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-05-04T12:16:08.355 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-04T12:16:08.464 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-04T12:16:08.574 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-04T12:16:08.592 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libcephfs2 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:08.726 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libradosstriper1 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:08.852 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-04T12:16:08.859 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-04T12:16:09.126 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-04T12:16:09.129 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up librbd1 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:09.255 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph-mgr-modules-core (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:09.381 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph-fuse (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:09.386 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-04T12:16:09.512 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-04T12:16:09.595 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-04T12:16:09.595 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-04T12:16:09.822 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-04T12:16:10.073 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-04T12:16:10.107 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libcephfs-dev (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:10.233 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-rados (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:10.341 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-04T12:16:10.367 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up librgw2 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:10.477 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up cephfs-top (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:10.578 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-04T12:16:10.592 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-04T12:16:10.851 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-04T12:16:11.119 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-04T12:16:11.245 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-04T12:16:11.538 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-04T12:16:11.664 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-04T12:16:11.731 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-rbd (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:11.882 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up rbd-fuse (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:11.957 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-04T12:16:12.008 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-rgw (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:12.135 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up python3-cephfs (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:12.235 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up cephfs-shell (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:12.274 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-04T12:16:12.336 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph-common (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:12.543 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-04T12:16:12.693 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-05-04T12:16:12.820 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-04T12:16:12.854 INFO:teuthology.orchestra.run.smithi111.stdout:Adding group ceph....done 2024-05-04T12:16:12.910 INFO:teuthology.orchestra.run.smithi052.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-04T12:16:13.155 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-04T12:16:13.231 INFO:teuthology.orchestra.run.smithi111.stdout:Adding system user ceph....done 2024-05-04T12:16:13.264 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-04T12:16:13.384 INFO:teuthology.orchestra.run.smithi111.stdout:Setting system user ceph properties....done 2024-05-04T12:16:13.401 INFO:teuthology.orchestra.run.smithi111.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-04T12:16:13.535 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-04T12:16:13.657 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-04T12:16:13.831 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-04T12:16:13.900 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up lua-any (27ubuntu1) ... 2024-05-04T12:16:14.026 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-04T12:16:14.327 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-04T12:16:14.453 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-04T12:16:14.556 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up cephfs-mirror (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:14.780 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-04T12:16:14.799 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-04T12:16:14.799 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-04T12:16:15.072 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-04T12:16:15.332 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-04T12:16:15.477 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph-test (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:15.518 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-04T12:16:15.579 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up radosgw (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:15.795 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-04T12:16:16.054 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-04T12:16:16.054 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-04T12:16:16.071 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-04T12:16:16.339 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-04T12:16:16.473 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph-base (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:16.724 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-04T12:16:16.820 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-04T12:16:17.017 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-04T12:16:17.300 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph-mds (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:17.361 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-04T12:16:17.487 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up librados2 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:17.502 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-04T12:16:17.502 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-04T12:16:17.614 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:17.731 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-05-04T12:16:18.002 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph-mgr (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:18.034 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libcephfs2 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:18.160 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libradosstriper1 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:18.209 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-04T12:16:18.209 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-04T12:16:18.286 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-04T12:16:18.571 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up librbd1 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:18.695 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph-osd (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:18.697 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mgr-modules-core (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:18.815 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-fuse (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:18.966 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-04T12:16:18.966 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-04T12:16:19.026 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-04T12:16:19.026 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-04T12:16:19.464 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:19.500 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libcephfs-dev (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:19.615 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:19.617 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-rados (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:19.743 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up librgw2 (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:19.766 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph-mon (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:19.869 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up cephfs-top (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:19.979 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-04T12:16:19.979 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-04T12:16:19.995 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-04T12:16:20.484 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph-mgr-cephadm (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:20.627 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:20.728 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph-mgr-dashboard (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:20.879 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ceph-volume (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:21.232 INFO:teuthology.orchestra.run.smithi111.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-04T12:16:21.257 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-rbd (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:21.341 INFO:teuthology.orchestra.run.smithi111.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-04T12:16:21.392 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up rbd-fuse (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:21.493 INFO:teuthology.orchestra.run.smithi111.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-04T12:16:21.518 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-rgw (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:21.644 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-cephfs (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:21.770 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up cephfs-shell (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:21.888 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-common (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:22.648 INFO:teuthology.orchestra.run.smithi052.stdout:Adding group ceph....done 2024-05-04T12:16:22.661 INFO:teuthology.orchestra.run.smithi111.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-04T12:16:22.991 INFO:teuthology.orchestra.run.smithi052.stdout:Adding system user ceph....done 2024-05-04T12:16:23.159 INFO:teuthology.orchestra.run.smithi052.stdout:Setting system user ceph properties....done 2024-05-04T12:16:23.177 INFO:teuthology.orchestra.run.smithi052.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-04T12:16:23.297 INFO:teuthology.orchestra.run.smithi111.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-04T12:16:23.300 DEBUG:teuthology.orchestra.run.smithi111:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-04T12:16:23.321 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-04T12:16:23.354 INFO:teuthology.orchestra.run.smithi111.stdout:Reading package lists... 2024-05-04T12:16:23.553 INFO:teuthology.orchestra.run.smithi111.stdout:Building dependency tree... 2024-05-04T12:16:23.554 INFO:teuthology.orchestra.run.smithi111.stdout:Reading state information... 2024-05-04T12:16:23.621 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-04T12:16:23.745 INFO:teuthology.orchestra.run.smithi111.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-05-04T12:16:23.745 INFO:teuthology.orchestra.run.smithi111.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-05-04T12:16:23.745 INFO:teuthology.orchestra.run.smithi111.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-05-04T12:16:23.745 INFO:teuthology.orchestra.run.smithi111.stdout:The following packages were automatically installed and are no longer required: 2024-05-04T12:16:23.745 INFO:teuthology.orchestra.run.smithi111.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-04T12:16:23.745 INFO:teuthology.orchestra.run.smithi111.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-04T12:16:23.745 INFO:teuthology.orchestra.run.smithi111.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-04T12:16:23.746 INFO:teuthology.orchestra.run.smithi111.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-04T12:16:23.746 INFO:teuthology.orchestra.run.smithi111.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-04T12:16:23.746 INFO:teuthology.orchestra.run.smithi111.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-04T12:16:23.747 INFO:teuthology.orchestra.run.smithi111.stdout:The following additional packages will be installed: 2024-05-04T12:16:23.747 INFO:teuthology.orchestra.run.smithi111.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-05-04T12:16:23.747 INFO:teuthology.orchestra.run.smithi111.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-05-04T12:16:23.748 INFO:teuthology.orchestra.run.smithi111.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-05-04T12:16:23.748 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-05-04T12:16:23.748 INFO:teuthology.orchestra.run.smithi111.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-04T12:16:23.749 INFO:teuthology.orchestra.run.smithi111.stdout:Suggested packages: 2024-05-04T12:16:23.749 INFO:teuthology.orchestra.run.smithi111.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-05-04T12:16:23.749 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-05-04T12:16:23.749 INFO:teuthology.orchestra.run.smithi111.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-05-04T12:16:23.793 INFO:teuthology.orchestra.run.smithi111.stdout:The following NEW packages will be installed: 2024-05-04T12:16:23.794 INFO:teuthology.orchestra.run.smithi111.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-05-04T12:16:23.794 INFO:teuthology.orchestra.run.smithi111.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-05-04T12:16:23.794 INFO:teuthology.orchestra.run.smithi111.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-05-04T12:16:23.794 INFO:teuthology.orchestra.run.smithi111.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-05-04T12:16:23.794 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-05-04T12:16:23.795 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-05-04T12:16:23.795 INFO:teuthology.orchestra.run.smithi111.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-04T12:16:23.796 INFO:teuthology.orchestra.run.smithi111.stdout:The following packages will be upgraded: 2024-05-04T12:16:23.796 INFO:teuthology.orchestra.run.smithi111.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-05-04T12:16:23.855 INFO:teuthology.orchestra.run.smithi111.stdout:4 upgraded, 39 newly installed, 0 to remove and 299 not upgraded. 2024-05-04T12:16:23.855 INFO:teuthology.orchestra.run.smithi111.stdout:Need to get 55.7 MB of archives. 2024-05-04T12:16:23.855 INFO:teuthology.orchestra.run.smithi111.stdout:After this operation, 202 MB of additional disk space will be used. 2024-05-04T12:16:23.855 INFO:teuthology.orchestra.run.smithi111.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-05-04T12:16:24.012 INFO:teuthology.orchestra.run.smithi111.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-05-04T12:16:24.020 INFO:teuthology.orchestra.run.smithi111.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-05-04T12:16:24.059 INFO:teuthology.orchestra.run.smithi111.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-05-04T12:16:24.060 INFO:teuthology.orchestra.run.smithi111.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-05-04T12:16:24.060 INFO:teuthology.orchestra.run.smithi111.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-05-04T12:16:24.072 INFO:teuthology.orchestra.run.smithi111.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-05-04T12:16:24.072 INFO:teuthology.orchestra.run.smithi111.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-05-04T12:16:24.075 INFO:teuthology.orchestra.run.smithi111.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-05-04T12:16:24.076 INFO:teuthology.orchestra.run.smithi111.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-05-04T12:16:24.076 INFO:teuthology.orchestra.run.smithi111.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-05-04T12:16:24.079 INFO:teuthology.orchestra.run.smithi111.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-05-04T12:16:24.080 INFO:teuthology.orchestra.run.smithi111.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-05-04T12:16:24.097 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up cephfs-mirror (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:24.100 INFO:teuthology.orchestra.run.smithi111.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-05-04T12:16:24.101 INFO:teuthology.orchestra.run.smithi111.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-05-04T12:16:24.102 INFO:teuthology.orchestra.run.smithi111.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-05-04T12:16:24.329 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-04T12:16:24.329 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-04T12:16:24.454 INFO:teuthology.orchestra.run.smithi111.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-05-04T12:16:24.456 INFO:teuthology.orchestra.run.smithi111.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-05-04T12:16:24.456 INFO:teuthology.orchestra.run.smithi111.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-05-04T12:16:24.459 INFO:teuthology.orchestra.run.smithi111.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-05-04T12:16:24.459 INFO:teuthology.orchestra.run.smithi111.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-05-04T12:16:24.464 INFO:teuthology.orchestra.run.smithi111.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-05-04T12:16:24.466 INFO:teuthology.orchestra.run.smithi111.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-05-04T12:16:24.474 INFO:teuthology.orchestra.run.smithi111.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-05-04T12:16:24.474 INFO:teuthology.orchestra.run.smithi111.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-05-04T12:16:24.497 INFO:teuthology.orchestra.run.smithi111.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-05-04T12:16:24.497 INFO:teuthology.orchestra.run.smithi111.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-05-04T12:16:24.512 INFO:teuthology.orchestra.run.smithi111.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-05-04T12:16:24.540 INFO:teuthology.orchestra.run.smithi111.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-05-04T12:16:24.545 INFO:teuthology.orchestra.run.smithi111.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-05-04T12:16:24.545 INFO:teuthology.orchestra.run.smithi111.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-05-04T12:16:24.546 INFO:teuthology.orchestra.run.smithi111.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-05-04T12:16:24.546 INFO:teuthology.orchestra.run.smithi111.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-05-04T12:16:24.561 INFO:teuthology.orchestra.run.smithi111.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-05-04T12:16:24.563 INFO:teuthology.orchestra.run.smithi111.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-05-04T12:16:24.798 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-test (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:24.818 INFO:teuthology.orchestra.run.smithi111.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-05-04T12:16:24.818 INFO:teuthology.orchestra.run.smithi111.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-05-04T12:16:24.818 INFO:teuthology.orchestra.run.smithi111.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-05-04T12:16:24.818 INFO:teuthology.orchestra.run.smithi111.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-05-04T12:16:24.819 INFO:teuthology.orchestra.run.smithi111.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-05-04T12:16:24.819 INFO:teuthology.orchestra.run.smithi111.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-05-04T12:16:24.830 INFO:teuthology.orchestra.run.smithi111.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-05-04T12:16:24.831 INFO:teuthology.orchestra.run.smithi111.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-05-04T12:16:24.925 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up radosgw (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:25.229 INFO:teuthology.orchestra.run.smithi111.stdout:Preconfiguring packages ... 2024-05-04T12:16:25.384 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-04T12:16:25.384 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-04T12:16:25.414 INFO:teuthology.orchestra.run.smithi111.stdout:Fetched 55.7 MB in 1s (54.3 MB/s) 2024-05-04T12:16:25.715 INFO:teuthology.orchestra.run.smithi111.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-05-04T12:16:25.720 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-04T12:16:25.818 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-base (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:25.859 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-04T12:16:26.174 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-04T12:16:26.472 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-05-04T12:16:26.645 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mds (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:26.884 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-04T12:16:26.884 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-04T12:16:27.347 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mgr (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:27.422 INFO:teuthology.orchestra.run.smithi111.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-05-04T12:16:27.560 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-04T12:16:27.560 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-04T12:16:27.689 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package flex. 2024-05-04T12:16:27.782 INFO:teuthology.orchestra.run.smithi111.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-05-04T12:16:27.788 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-05-04T12:16:27.829 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking flex (2.6.4-8build2) ... 2024-05-04T12:16:28.032 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-osd (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:28.309 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-04T12:16:28.309 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-04T12:16:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-04T12:16:28.518 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-05-04T12:16:28.759 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:28.885 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:28.946 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-04T12:16:29.036 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mon (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:29.247 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-04T12:16:29.247 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-04T12:16:29.448 INFO:teuthology.orchestra.run.smithi111.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-05-04T12:16:29.453 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-04T12:16:29.593 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-04T12:16:29.712 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mgr-cephadm (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:29.855 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:29.932 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-04T12:16:29.981 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mgr-dashboard (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:30.058 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-04T12:16:30.132 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-volume (19.0.0-2455-g09dbd6bb-1jammy) ... 2024-05-04T12:16:30.393 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-04T12:16:30.510 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-04T12:16:30.611 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-04T12:16:30.755 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-04T12:16:30.792 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package bison. 2024-05-04T12:16:30.897 INFO:teuthology.orchestra.run.smithi111.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-05-04T12:16:30.903 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-05-04T12:16:30.953 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-05-04T12:16:31.387 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-04T12:16:31.415 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-05-04T12:16:31.454 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-05-04T12:16:31.740 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package dump. 2024-05-04T12:16:31.768 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-05-04T12:16:31.807 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking dump (0.4b47-3) ... 2024-05-04T12:16:31.939 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-04T12:16:32.094 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package indent. 2024-05-04T12:16:32.110 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-05-04T12:16:32.152 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-04T12:16:32.414 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-04T12:16:32.429 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-05-04T12:16:32.472 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-04T12:16:32.577 INFO:teuthology.orchestra.run.smithi052.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-04T12:16:32.580 DEBUG:teuthology.orchestra.run.smithi052:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-04T12:16:32.643 INFO:teuthology.orchestra.run.smithi052.stdout:Reading package lists... 2024-05-04T12:16:32.751 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-04T12:16:32.778 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-05-04T12:16:32.817 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-04T12:16:32.847 INFO:teuthology.orchestra.run.smithi052.stdout:Building dependency tree... 2024-05-04T12:16:32.848 INFO:teuthology.orchestra.run.smithi052.stdout:Reading state information... 2024-05-04T12:16:33.051 INFO:teuthology.orchestra.run.smithi052.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-05-04T12:16:33.051 INFO:teuthology.orchestra.run.smithi052.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-05-04T12:16:33.051 INFO:teuthology.orchestra.run.smithi052.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-05-04T12:16:33.051 INFO:teuthology.orchestra.run.smithi052.stdout:The following packages were automatically installed and are no longer required: 2024-05-04T12:16:33.051 INFO:teuthology.orchestra.run.smithi052.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-04T12:16:33.051 INFO:teuthology.orchestra.run.smithi052.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-04T12:16:33.051 INFO:teuthology.orchestra.run.smithi052.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-04T12:16:33.052 INFO:teuthology.orchestra.run.smithi052.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-04T12:16:33.052 INFO:teuthology.orchestra.run.smithi052.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-04T12:16:33.052 INFO:teuthology.orchestra.run.smithi052.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-04T12:16:33.052 INFO:teuthology.orchestra.run.smithi052.stdout:The following additional packages will be installed: 2024-05-04T12:16:33.053 INFO:teuthology.orchestra.run.smithi052.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-05-04T12:16:33.053 INFO:teuthology.orchestra.run.smithi052.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-05-04T12:16:33.053 INFO:teuthology.orchestra.run.smithi052.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-05-04T12:16:33.053 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-05-04T12:16:33.053 INFO:teuthology.orchestra.run.smithi052.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-04T12:16:33.055 INFO:teuthology.orchestra.run.smithi052.stdout:Suggested packages: 2024-05-04T12:16:33.055 INFO:teuthology.orchestra.run.smithi052.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-05-04T12:16:33.055 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-05-04T12:16:33.055 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-05-04T12:16:33.095 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-04T12:16:33.098 INFO:teuthology.orchestra.run.smithi052.stdout:The following NEW packages will be installed: 2024-05-04T12:16:33.098 INFO:teuthology.orchestra.run.smithi052.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-05-04T12:16:33.098 INFO:teuthology.orchestra.run.smithi052.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-05-04T12:16:33.099 INFO:teuthology.orchestra.run.smithi052.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-05-04T12:16:33.099 INFO:teuthology.orchestra.run.smithi052.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-05-04T12:16:33.099 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-05-04T12:16:33.100 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-05-04T12:16:33.100 INFO:teuthology.orchestra.run.smithi052.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-04T12:16:33.101 INFO:teuthology.orchestra.run.smithi052.stdout:The following packages will be upgraded: 2024-05-04T12:16:33.101 INFO:teuthology.orchestra.run.smithi052.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-05-04T12:16:33.112 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-05-04T12:16:33.153 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-05-04T12:16:33.284 INFO:teuthology.orchestra.run.smithi052.stdout:4 upgraded, 39 newly installed, 0 to remove and 299 not upgraded. 2024-05-04T12:16:33.284 INFO:teuthology.orchestra.run.smithi052.stdout:Need to get 55.7 MB of archives. 2024-05-04T12:16:33.284 INFO:teuthology.orchestra.run.smithi052.stdout:After this operation, 202 MB of additional disk space will be used. 2024-05-04T12:16:33.285 INFO:teuthology.orchestra.run.smithi052.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-05-04T12:16:33.490 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libfl2:amd64. 2024-05-04T12:16:33.518 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-05-04T12:16:33.557 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-05-04T12:16:33.776 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-05-04T12:16:33.792 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-05-04T12:16:33.826 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-04T12:16:33.916 INFO:teuthology.orchestra.run.smithi052.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-05-04T12:16:33.933 INFO:teuthology.orchestra.run.smithi052.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-05-04T12:16:34.057 INFO:teuthology.orchestra.run.smithi052.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-05-04T12:16:34.059 INFO:teuthology.orchestra.run.smithi052.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-05-04T12:16:34.060 INFO:teuthology.orchestra.run.smithi052.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-05-04T12:16:34.071 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libjson-perl. 2024-05-04T12:16:34.087 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-05-04T12:16:34.088 INFO:teuthology.orchestra.run.smithi052.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-05-04T12:16:34.090 INFO:teuthology.orchestra.run.smithi052.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-05-04T12:16:34.094 INFO:teuthology.orchestra.run.smithi052.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-05-04T12:16:34.096 INFO:teuthology.orchestra.run.smithi052.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-05-04T12:16:34.097 INFO:teuthology.orchestra.run.smithi052.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-05-04T12:16:34.129 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-05-04T12:16:34.263 INFO:teuthology.orchestra.run.smithi052.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-05-04T12:16:34.424 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libllvm14:amd64. 2024-05-04T12:16:34.452 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-05-04T12:16:34.490 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-04T12:16:34.555 INFO:teuthology.orchestra.run.smithi052.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-05-04T12:16:34.565 INFO:teuthology.orchestra.run.smithi052.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-05-04T12:16:34.570 INFO:teuthology.orchestra.run.smithi052.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-05-04T12:16:34.655 INFO:teuthology.orchestra.run.smithi052.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-05-04T12:16:35.779 INFO:teuthology.orchestra.run.smithi052.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-05-04T12:16:35.784 INFO:teuthology.orchestra.run.smithi052.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-05-04T12:16:35.784 INFO:teuthology.orchestra.run.smithi052.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-05-04T12:16:35.791 INFO:teuthology.orchestra.run.smithi052.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-05-04T12:16:35.791 INFO:teuthology.orchestra.run.smithi052.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-05-04T12:16:35.802 INFO:teuthology.orchestra.run.smithi052.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-05-04T12:16:35.969 INFO:teuthology.orchestra.run.smithi052.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-05-04T12:16:36.185 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-04T12:16:36.213 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-04T12:16:36.243 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-04T12:16:36.505 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-04T12:16:36.532 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-04T12:16:36.558 INFO:teuthology.orchestra.run.smithi052.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-05-04T12:16:36.559 INFO:teuthology.orchestra.run.smithi052.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-05-04T12:16:36.571 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-04T12:16:36.738 INFO:teuthology.orchestra.run.smithi052.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-05-04T12:16:36.738 INFO:teuthology.orchestra.run.smithi052.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-05-04T12:16:36.776 INFO:teuthology.orchestra.run.smithi052.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-05-04T12:16:36.834 INFO:teuthology.orchestra.run.smithi052.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-05-04T12:16:36.840 INFO:teuthology.orchestra.run.smithi052.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-05-04T12:16:36.841 INFO:teuthology.orchestra.run.smithi052.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-05-04T12:16:36.842 INFO:teuthology.orchestra.run.smithi052.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-05-04T12:16:36.843 INFO:teuthology.orchestra.run.smithi052.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-05-04T12:16:36.890 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-04T12:16:36.918 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-04T12:16:36.957 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-04T12:16:37.053 INFO:teuthology.orchestra.run.smithi052.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-05-04T12:16:37.277 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-04T12:16:37.305 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-05-04T12:16:37.344 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-04T12:16:37.464 INFO:teuthology.orchestra.run.smithi052.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-05-04T12:16:37.647 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-04T12:16:37.674 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-05-04T12:16:37.713 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-04T12:16:38.108 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-04T12:16:38.136 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-04T12:16:38.183 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-04T12:16:38.295 INFO:teuthology.orchestra.run.smithi052.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-05-04T12:16:38.295 INFO:teuthology.orchestra.run.smithi052.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-05-04T12:16:38.295 INFO:teuthology.orchestra.run.smithi052.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-05-04T12:16:38.295 INFO:teuthology.orchestra.run.smithi052.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-05-04T12:16:38.296 INFO:teuthology.orchestra.run.smithi052.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-05-04T12:16:38.296 INFO:teuthology.orchestra.run.smithi052.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-05-04T12:16:38.325 INFO:teuthology.orchestra.run.smithi052.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-05-04T12:16:38.326 INFO:teuthology.orchestra.run.smithi052.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-05-04T12:16:38.530 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-04T12:16:38.558 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-04T12:16:38.596 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-04T12:16:38.752 INFO:teuthology.orchestra.run.smithi052.stdout:Preconfiguring packages ... 2024-05-04T12:16:38.909 INFO:teuthology.orchestra.run.smithi052.stdout:Fetched 55.7 MB in 5s (10.7 MB/s) 2024-05-04T12:16:38.966 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-04T12:16:38.993 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-04T12:16:39.041 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-04T12:16:39.231 INFO:teuthology.orchestra.run.smithi052.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-05-04T12:16:39.237 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-04T12:16:39.309 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-04T12:16:39.326 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-04T12:16:39.368 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-04T12:16:39.380 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-04T12:16:39.881 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-04T12:16:39.908 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-05-04T12:16:39.947 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-05-04T12:16:40.038 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-05-04T12:16:40.384 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package modemmanager. 2024-05-04T12:16:40.411 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-04T12:16:40.458 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-04T12:16:40.971 INFO:teuthology.orchestra.run.smithi052.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-05-04T12:16:41.099 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package network-manager. 2024-05-04T12:16:41.127 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-05-04T12:16:41.166 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-05-04T12:16:41.237 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package flex. 2024-05-04T12:16:41.344 INFO:teuthology.orchestra.run.smithi052.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-05-04T12:16:41.350 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-05-04T12:16:41.394 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking flex (2.6.4-8build2) ... 2024-05-04T12:16:41.832 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package ppp. 2024-05-04T12:16:41.859 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-05-04T12:16:41.927 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-04T12:16:42.083 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-05-04T12:16:42.135 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-05-04T12:16:42.453 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package pptp-linux. 2024-05-04T12:16:42.481 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-05-04T12:16:42.512 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-04T12:16:42.519 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-05-04T12:16:42.848 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-04T12:16:42.875 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-05-04T12:16:42.922 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-05-04T12:16:43.091 INFO:teuthology.orchestra.run.smithi052.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-05-04T12:16:43.097 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-04T12:16:43.225 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-04T12:16:43.242 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-04T12:16:43.270 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-05-04T12:16:43.309 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking postgresql-client-common (238) ... 2024-05-04T12:16:43.581 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-04T12:16:43.621 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package postgresql-client-14. 2024-05-04T12:16:43.650 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-04T12:16:43.688 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-04T12:16:43.723 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-04T12:16:44.059 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-04T12:16:44.251 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package postgresql-common. 2024-05-04T12:16:44.279 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-05-04T12:16:44.335 INFO:teuthology.orchestra.run.smithi111.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-04T12:16:44.366 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking postgresql-common (238) ... 2024-05-04T12:16:44.407 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package bison. 2024-05-04T12:16:44.515 INFO:teuthology.orchestra.run.smithi052.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-05-04T12:16:44.521 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-05-04T12:16:44.576 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-05-04T12:16:44.817 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package postgresql-14. 2024-05-04T12:16:44.845 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-04T12:16:44.887 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-04T12:16:44.985 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-04T12:16:45.013 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-05-04T12:16:45.051 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-05-04T12:16:45.388 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package dump. 2024-05-04T12:16:45.416 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-05-04T12:16:45.455 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking dump (0.4b47-3) ... 2024-05-04T12:16:45.725 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package indent. 2024-05-04T12:16:45.742 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-05-04T12:16:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-04T12:16:46.011 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-04T12:16:46.028 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-05-04T12:16:46.069 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-04T12:16:46.398 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-04T12:16:46.427 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-05-04T12:16:46.465 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-04T12:16:46.735 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-04T12:16:46.751 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-05-04T12:16:46.793 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-05-04T12:16:46.945 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package postgresql. 2024-05-04T12:16:46.962 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-05-04T12:16:47.003 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking postgresql (14+238) ... 2024-05-04T12:16:47.055 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libfl2:amd64. 2024-05-04T12:16:47.071 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-05-04T12:16:47.105 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-05-04T12:16:47.239 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package postgresql-client. 2024-05-04T12:16:47.267 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-05-04T12:16:47.305 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking postgresql-client (14+238) ... 2024-05-04T12:16:47.382 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-05-04T12:16:47.411 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-05-04T12:16:47.474 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-04T12:16:47.533 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-04T12:16:47.561 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-05-04T12:16:47.599 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking postgresql-contrib (14+238) ... 2024-05-04T12:16:47.743 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libjson-perl. 2024-05-04T12:16:47.772 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-05-04T12:16:47.810 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-05-04T12:16:47.902 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package tcl8.6. 2024-05-04T12:16:47.930 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-05-04T12:16:47.968 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-04T12:16:48.198 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libllvm14:amd64. 2024-05-04T12:16:48.212 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package tcl. 2024-05-04T12:16:48.226 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-05-04T12:16:48.229 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-05-04T12:16:48.265 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-04T12:16:48.276 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-05-04T12:16:48.616 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package wpasupplicant. 2024-05-04T12:16:48.644 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-05-04T12:16:48.682 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-04T12:16:49.029 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-04T12:16:49.057 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-05-04T12:16:49.095 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-05-04T12:16:49.482 INFO:teuthology.orchestra.run.smithi111.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-04T12:16:49.510 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-05-04T12:16:49.556 INFO:teuthology.orchestra.run.smithi111.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-04T12:16:49.837 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up postgresql-client-common (238) ... 2024-05-04T12:16:49.934 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-04T12:16:49.962 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-04T12:16:50.001 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-04T12:16:50.071 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up flex (2.6.4-8build2) ... 2024-05-04T12:16:50.197 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-04T12:16:50.237 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-04T12:16:50.251 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-04T12:16:50.287 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-04T12:16:50.323 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-04T12:16:50.466 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-05-04T12:16:50.598 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-04T12:16:50.609 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-04T12:16:50.626 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-04T12:16:50.664 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-04T12:16:50.735 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-04T12:16:51.018 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-04T12:16:51.046 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-05-04T12:16:51.084 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-04T12:16:51.209 INFO:teuthology.orchestra.run.smithi111.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-04T12:16:51.307 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-04T12:16:51.629 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-04T12:16:51.658 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-05-04T12:16:51.695 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-04T12:16:51.705 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-04T12:16:51.705 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-04T12:16:52.049 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-04T12:16:52.077 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-04T12:16:52.115 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-04T12:16:52.202 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-04T12:16:52.303 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-05-04T12:16:52.404 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-04T12:16:52.485 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-04T12:16:52.513 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-04T12:16:52.530 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-05-04T12:16:52.551 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-04T12:16:52.657 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-05-04T12:16:52.783 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-05-04T12:16:52.871 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-04T12:16:52.888 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-04T12:16:52.908 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-04T12:16:52.929 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-04T12:16:53.035 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-05-04T12:16:53.326 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-04T12:16:53.354 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-04T12:16:53.392 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-04T12:16:53.633 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-04T12:16:53.758 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up dump (0.4b47-3) ... 2024-05-04T12:16:53.851 INFO:teuthology.orchestra.run.smithi111.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-04T12:16:53.896 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-04T12:16:53.918 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-04T12:16:53.924 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-05-04T12:16:53.963 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-05-04T12:16:54.044 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-04T12:16:54.170 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up tcl (8.6.11+1build2) ... 2024-05-04T12:16:54.287 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libjson-perl (4.04000-1) ... 2024-05-04T12:16:54.324 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package modemmanager. 2024-05-04T12:16:54.352 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-04T12:16:54.399 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-04T12:16:54.413 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-05-04T12:16:54.488 INFO:teuthology.orchestra.run.smithi111.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-05-04T12:16:54.557 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-04T12:16:54.683 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-04T12:16:54.859 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-04T12:16:54.985 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-04T12:16:55.056 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package network-manager. 2024-05-04T12:16:55.085 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-05-04T12:16:55.111 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-05-04T12:16:55.132 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-05-04T12:16:55.262 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up postgresql-client (14+238) ... 2024-05-04T12:16:55.364 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-05-04T12:16:55.780 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ppp. 2024-05-04T12:16:55.809 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-05-04T12:16:55.848 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:16:55.848 INFO:teuthology.orchestra.run.smithi111.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-04T12:16:55.848 INFO:teuthology.orchestra.run.smithi111.stdout:which means they are currently configured by ifupdown: 2024-05-04T12:16:55.848 INFO:teuthology.orchestra.run.smithi111.stdout:- enp3s0f1 2024-05-04T12:16:55.848 INFO:teuthology.orchestra.run.smithi111.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-04T12:16:55.848 INFO:teuthology.orchestra.run.smithi111.stdout:remove their configuration from /etc/network/interfaces. 2024-05-04T12:16:55.848 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:16:55.966 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-04T12:16:56.075 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-05-04T12:16:56.306 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-04T12:16:56.377 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package pptp-linux. 2024-05-04T12:16:56.405 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-05-04T12:16:56.518 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-05-04T12:16:56.676 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-04T12:16:56.779 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-04T12:16:56.797 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-05-04T12:16:56.838 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-05-04T12:16:57.107 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-04T12:16:57.136 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-05-04T12:16:57.165 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking postgresql-client-common (238) ... 2024-05-04T12:16:57.241 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-04T12:16:57.368 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-04T12:16:57.469 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package postgresql-client-14. 2024-05-04T12:16:57.486 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-05-04T12:16:57.498 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-04T12:16:57.536 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-04T12:16:57.611 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up postgresql-common (238) ... 2024-05-04T12:16:58.133 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package postgresql-common. 2024-05-04T12:16:58.161 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-05-04T12:16:58.218 INFO:teuthology.orchestra.run.smithi052.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-04T12:16:58.247 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking postgresql-common (238) ... 2024-05-04T12:16:58.553 INFO:teuthology.orchestra.run.smithi111.stdout:Adding user postgres to group ssl-cert 2024-05-04T12:16:58.673 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package postgresql-14. 2024-05-04T12:16:58.702 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-04T12:16:58.743 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-04T12:16:58.813 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:16:58.813 INFO:teuthology.orchestra.run.smithi111.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-04T12:16:59.011 INFO:teuthology.orchestra.run.smithi111.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-04T12:16:59.011 INFO:teuthology.orchestra.run.smithi111.stdout:Removing obsolete dictionary files: 2024-05-04T12:16:59.554 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-04T12:16:59.931 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-04T12:16:59.959 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package postgresql. 2024-05-04T12:16:59.977 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-05-04T12:17:00.009 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking postgresql (14+238) ... 2024-05-04T12:17:00.236 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package postgresql-client. 2024-05-04T12:17:00.245 INFO:teuthology.orchestra.run.smithi111.stdout:Creating new PostgreSQL cluster 14/main ... 2024-05-04T12:17:00.265 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-05-04T12:17:00.272 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-05-04T12:17:00.303 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking postgresql-client (14+238) ... 2024-05-04T12:17:00.409 INFO:teuthology.orchestra.run.smithi111.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-04T12:17:00.409 INFO:teuthology.orchestra.run.smithi111.stdout:This user must also own the server process. 2024-05-04T12:17:00.409 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:00.409 INFO:teuthology.orchestra.run.smithi111.stdout:The database cluster will be initialized with locale "en_US". 2024-05-04T12:17:00.409 INFO:teuthology.orchestra.run.smithi111.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-04T12:17:00.409 INFO:teuthology.orchestra.run.smithi111.stdout:The default text search configuration will be set to "english". 2024-05-04T12:17:00.409 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:00.409 INFO:teuthology.orchestra.run.smithi111.stdout:Data page checksums are disabled. 2024-05-04T12:17:00.409 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:00.410 INFO:teuthology.orchestra.run.smithi111.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-05-04T12:17:00.410 INFO:teuthology.orchestra.run.smithi111.stdout:creating subdirectories ... ok 2024-05-04T12:17:00.410 INFO:teuthology.orchestra.run.smithi111.stdout:selecting dynamic shared memory implementation ... posix 2024-05-04T12:17:00.426 INFO:teuthology.orchestra.run.smithi111.stdout:selecting default max_connections ... 100 2024-05-04T12:17:00.447 INFO:teuthology.orchestra.run.smithi111.stdout:selecting default shared_buffers ... 128MB 2024-05-04T12:17:00.460 INFO:teuthology.orchestra.run.smithi111.stdout:selecting default time zone ... Etc/UTC 2024-05-04T12:17:00.461 INFO:teuthology.orchestra.run.smithi111.stdout:creating configuration files ... ok 2024-05-04T12:17:00.547 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-04T12:17:00.564 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-05-04T12:17:00.581 INFO:teuthology.orchestra.run.smithi111.stdout:running bootstrap script ... ok 2024-05-04T12:17:00.605 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking postgresql-contrib (14+238) ... 2024-05-04T12:17:00.916 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package tcl8.6. 2024-05-04T12:17:00.945 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-05-04T12:17:00.982 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-04T12:17:01.269 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package tcl. 2024-05-04T12:17:01.298 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-05-04T12:17:01.339 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-05-04T12:17:01.623 INFO:teuthology.orchestra.run.smithi111.stdout:performing post-bootstrap initialization ... ok 2024-05-04T12:17:01.697 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package wpasupplicant. 2024-05-04T12:17:01.726 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-05-04T12:17:01.763 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-04T12:17:02.118 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-04T12:17:02.135 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-05-04T12:17:02.176 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-05-04T12:17:02.351 INFO:teuthology.orchestra.run.smithi111.stdout:syncing data to disk ... ok 2024-05-04T12:17:02.487 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-04T12:17:02.516 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-05-04T12:17:02.554 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-04T12:17:02.735 INFO:teuthology.orchestra.run.smithi111.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-04T12:17:02.825 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up postgresql-client-common (238) ... 2024-05-04T12:17:03.043 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up flex (2.6.4-8build2) ... 2024-05-04T12:17:03.169 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-04T12:17:03.295 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-04T12:17:03.396 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-05-04T12:17:03.506 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-04T12:17:03.966 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-04T12:17:04.484 INFO:teuthology.orchestra.run.smithi052.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-04T12:17:04.554 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-04T12:17:04.917 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-04T12:17:04.917 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-04T12:17:05.383 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-04T12:17:05.467 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-04T12:17:05.508 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-05-04T12:17:05.568 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up postgresql-contrib (14+238) ... 2024-05-04T12:17:05.634 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-04T12:17:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up postgresql (14+238) ... 2024-05-04T12:17:05.760 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-05-04T12:17:05.820 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-04T12:17:05.886 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-05-04T12:17:05.946 INFO:teuthology.orchestra.run.smithi111.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-04T12:17:06.012 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-05-04T12:17:06.139 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-04T12:17:06.208 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-04T12:17:06.208 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-04T12:17:06.239 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-05-04T12:17:06.857 INFO:teuthology.orchestra.run.smithi111.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-04T12:17:06.886 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-04T12:17:07.012 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up dump (0.4b47-3) ... 2024-05-04T12:17:07.025 INFO:teuthology.orchestra.run.smithi111.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-04T12:17:07.106 INFO:teuthology.orchestra.run.smithi052.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-04T12:17:07.172 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-04T12:17:07.273 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-04T12:17:07.327 INFO:teuthology.orchestra.run.smithi111.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-04T12:17:07.391 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up tcl (8.6.11+1build2) ... 2024-05-04T12:17:07.471 INFO:teuthology.orchestra.run.smithi111.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-04T12:17:07.517 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libjson-perl (4.04000-1) ... 2024-05-04T12:17:07.643 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-05-04T12:17:07.733 INFO:teuthology.orchestra.run.smithi052.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-05-04T12:17:07.803 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-04T12:17:07.929 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-04T12:17:08.147 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-04T12:17:08.248 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-04T12:17:08.349 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-05-04T12:17:08.525 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up postgresql-client (14+238) ... 2024-05-04T12:17:08.605 INFO:teuthology.orchestra.run.smithi111.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-05-04T12:17:08.651 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-05-04T12:17:08.697 INFO:teuthology.orchestra.run.smithi111.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-04T12:17:09.130 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:09.130 INFO:teuthology.orchestra.run.smithi052.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-04T12:17:09.131 INFO:teuthology.orchestra.run.smithi052.stdout:which means they are currently configured by ifupdown: 2024-05-04T12:17:09.131 INFO:teuthology.orchestra.run.smithi052.stdout:- ens1f0 2024-05-04T12:17:09.131 INFO:teuthology.orchestra.run.smithi052.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-04T12:17:09.131 INFO:teuthology.orchestra.run.smithi052.stdout:remove their configuration from /etc/network/interfaces. 2024-05-04T12:17:09.131 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:09.197 INFO:teuthology.orchestra.run.smithi111.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-04T12:17:09.202 DEBUG:teuthology.parallel:result is None 2024-05-04T12:17:09.247 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-04T12:17:09.561 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-04T12:17:09.877 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-04T12:17:10.337 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-04T12:17:10.464 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-04T12:17:10.623 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-05-04T12:17:10.749 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up postgresql-common (238) ... 2024-05-04T12:17:11.672 INFO:teuthology.orchestra.run.smithi052.stdout:Adding user postgres to group ssl-cert 2024-05-04T12:17:11.898 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:11.898 INFO:teuthology.orchestra.run.smithi052.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-04T12:17:12.085 INFO:teuthology.orchestra.run.smithi052.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-04T12:17:12.085 INFO:teuthology.orchestra.run.smithi052.stdout:Removing obsolete dictionary files: 2024-05-04T12:17:12.656 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-04T12:17:13.017 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-04T12:17:13.352 INFO:teuthology.orchestra.run.smithi052.stdout:Creating new PostgreSQL cluster 14/main ... 2024-05-04T12:17:13.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-05-04T12:17:13.410 INFO:teuthology.orchestra.run.smithi052.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-04T12:17:13.410 INFO:teuthology.orchestra.run.smithi052.stdout:This user must also own the server process. 2024-05-04T12:17:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:13.410 INFO:teuthology.orchestra.run.smithi052.stdout:The database cluster will be initialized with locale "en_US". 2024-05-04T12:17:13.411 INFO:teuthology.orchestra.run.smithi052.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-04T12:17:13.411 INFO:teuthology.orchestra.run.smithi052.stdout:The default text search configuration will be set to "english". 2024-05-04T12:17:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:13.411 INFO:teuthology.orchestra.run.smithi052.stdout:Data page checksums are disabled. 2024-05-04T12:17:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:13.411 INFO:teuthology.orchestra.run.smithi052.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-05-04T12:17:13.411 INFO:teuthology.orchestra.run.smithi052.stdout:creating subdirectories ... ok 2024-05-04T12:17:13.411 INFO:teuthology.orchestra.run.smithi052.stdout:selecting dynamic shared memory implementation ... posix 2024-05-04T12:17:13.427 INFO:teuthology.orchestra.run.smithi052.stdout:selecting default max_connections ... 100 2024-05-04T12:17:13.445 INFO:teuthology.orchestra.run.smithi052.stdout:selecting default shared_buffers ... 128MB 2024-05-04T12:17:13.460 INFO:teuthology.orchestra.run.smithi052.stdout:selecting default time zone ... Etc/UTC 2024-05-04T12:17:13.461 INFO:teuthology.orchestra.run.smithi052.stdout:creating configuration files ... ok 2024-05-04T12:17:13.583 INFO:teuthology.orchestra.run.smithi052.stdout:running bootstrap script ... ok 2024-05-04T12:17:14.237 INFO:teuthology.orchestra.run.smithi052.stdout:performing post-bootstrap initialization ... ok 2024-05-04T12:17:14.836 INFO:teuthology.orchestra.run.smithi052.stdout:syncing data to disk ... ok 2024-05-04T12:17:15.216 INFO:teuthology.orchestra.run.smithi052.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-04T12:17:17.965 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-04T12:17:18.091 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up postgresql-contrib (14+238) ... 2024-05-04T12:17:18.242 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up postgresql (14+238) ... 2024-05-04T12:17:18.368 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-04T12:17:18.494 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-04T12:17:18.738 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-04T12:17:18.739 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-04T12:17:19.263 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-04T12:17:19.348 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-04T12:17:19.633 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-04T12:17:19.794 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-04T12:17:21.230 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-05-04T12:17:21.338 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-04T12:17:21.809 INFO:teuthology.orchestra.run.smithi052.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-04T12:17:21.814 DEBUG:teuthology.parallel:result is None 2024-05-04T12:17:21.814 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=09dbd6bba52d19ba471c33bc15c008e6ad158ea6 2024-05-04T12:17:21.954 DEBUG:teuthology.orchestra.run.smithi052:> dpkg-query -W -f '${Version}' ceph 2024-05-04T12:17:21.974 INFO:teuthology.orchestra.run.smithi052.stdout:19.0.0-2455-g09dbd6bb-1jammy 2024-05-04T12:17:21.975 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2455-g09dbd6bb-1jammy 2024-05-04T12:17:21.975 INFO:teuthology.task.install:The correct ceph version 19.0.0-2455-g09dbd6bb-1jammy is installed. 2024-05-04T12:17:21.976 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=09dbd6bba52d19ba471c33bc15c008e6ad158ea6 2024-05-04T12:17:22.193 DEBUG:teuthology.orchestra.run.smithi111:> dpkg-query -W -f '${Version}' ceph 2024-05-04T12:17:22.213 INFO:teuthology.orchestra.run.smithi111.stdout:19.0.0-2455-g09dbd6bb-1jammy 2024-05-04T12:17:22.213 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2455-g09dbd6bb-1jammy 2024-05-04T12:17:22.213 INFO:teuthology.task.install:The correct ceph version 19.0.0-2455-g09dbd6bb-1jammy is installed. 2024-05-04T12:17:22.215 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-04T12:17:22.215 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:22.215 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-04T12:17:22.230 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:22.230 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-04T12:17:22.271 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-04T12:17:22.271 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:22.271 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/daemon-helper 2024-05-04T12:17:22.285 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-04T12:17:22.341 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:22.341 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2024-05-04T12:17:22.354 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-04T12:17:22.409 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-04T12:17:22.410 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:22.410 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-04T12:17:22.423 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-04T12:17:22.477 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:22.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-04T12:17:22.490 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-04T12:17:22.544 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-04T12:17:22.545 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:22.545 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/stdin-killer 2024-05-04T12:17:22.558 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-04T12:17:22.614 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:22.615 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2024-05-04T12:17:22.627 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-04T12:17:22.681 INFO:teuthology.run_tasks:Running task ceph... 2024-05-04T12:17:22.759 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-04T12:17:22.759 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 777 /var/log/ceph 2024-05-04T12:17:22.761 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /var/log/ceph 2024-05-04T12:17:22.774 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-04T12:17:22.774 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-04T12:17:22.815 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-04T12:17:22.829 INFO:tasks.ceph:Creating extra log directories... 2024-05-04T12:17:22.829 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-04T12:17:22.870 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-04T12:17:22.888 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-04T12:17:22.888 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-04T12:17:22.889 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-04T12:17:22.889 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:22.889 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-04T12:17:22.929 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-04T12:17:23.007 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:23.007 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-04T12:17:23.012 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-04T12:17:23.090 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-04T12:17:23.090 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-04T12:17:23.090 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278', 'branch': 'wip-yuri4-testing-2024-04-29-0642', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev}', 'email': None, 'first_in_suite': False, 'job_id': '7686278', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri4-testing-2024-04-29-0642'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '09dbd6bba52d19ba471c33bc15c008e6ad158ea6'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '09dbd6bba52d19ba471c33bc15c008e6ad158ea6'}}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri4-testing-2024-04-29-0642', 'sha1': '09dbd6bba52d19ba471c33bc15c008e6ad158ea6'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1311, 'sha1': '09dbd6bba52d19ba471c33bc15c008e6ad158ea6', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'fs', 'suite_branch': 'wip-yuri4-testing-2024-04-29-0642', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_09dbd6bba52d19ba471c33bc15c008e6ad158ea6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '09dbd6bba52d19ba471c33bc15c008e6ad158ea6', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi052.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDPZERTOJNDZxL2XAJILk+M8iDgy2uK9e+uQScedQDF0HhdKaCNtClQgSVnFoUD/Cy1/bMoA8+vv4Y39DyglmFA=', 'smithi111.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM9FpB/ROanvtTByjSaVIgOKZcrhO6T9+F6tFHY31AHl+5ybolof2Jmzat+7DOV3/ui3LkwWN3sRAFnTRfk2HaY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '09dbd6bba52d19ba471c33bc15c008e6ad158ea6'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '09dbd6bba52d19ba471c33bc15c008e6ad158ea6', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-02_19:10:09', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-04T12:17:23.091 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-04T12:17:23.093 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-04T12:17:23.134 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m0777 -- /var/run/ceph 2024-05-04T12:17:23.142 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m0777 -- /var/run/ceph 2024-05-04T12:17:23.190 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:23.190 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/scratch_devs of=/dev/stdout 2024-05-04T12:17:23.201 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-04T12:17:23.201 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_1 2024-05-04T12:17:23.249 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-04T12:17:23.250 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T12:17:23.250 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 876 Links: 1 2024-05-04T12:17:23.250 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T12:17:23.250 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-05-04 12:11:12.122264747 +0000 2024-05-04T12:17:23.250 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-05-04 12:11:11.858267105 +0000 2024-05-04T12:17:23.250 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-05-04 12:11:11.858267105 +0000 2024-05-04T12:17:23.250 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: - 2024-05-04T12:17:23.250 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-04T12:17:23.304 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-05-04T12:17:23.304 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-05-04T12:17:23.304 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000311381 s, 1.6 MB/s 2024-05-04T12:17:23.305 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-04T12:17:23.355 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_2 2024-05-04T12:17:23.405 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-04T12:17:23.405 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T12:17:23.406 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 896 Links: 1 2024-05-04T12:17:23.406 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T12:17:23.406 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-05-04 12:11:12.370262532 +0000 2024-05-04T12:17:23.406 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-05-04 12:11:12.366262568 +0000 2024-05-04T12:17:23.406 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-05-04 12:11:12.366262568 +0000 2024-05-04T12:17:23.406 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: - 2024-05-04T12:17:23.406 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-04T12:17:23.459 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-05-04T12:17:23.460 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-05-04T12:17:23.460 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000204437 s, 2.5 MB/s 2024-05-04T12:17:23.461 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-04T12:17:23.511 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_3 2024-05-04T12:17:23.558 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-04T12:17:23.558 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T12:17:23.558 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 914 Links: 1 2024-05-04T12:17:23.558 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T12:17:23.558 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-05-04 12:11:13.118255850 +0000 2024-05-04T12:17:23.558 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-05-04 12:11:12.838258352 +0000 2024-05-04T12:17:23.558 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-05-04 12:11:12.838258352 +0000 2024-05-04T12:17:23.558 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: - 2024-05-04T12:17:23.558 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-04T12:17:23.612 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-05-04T12:17:23.612 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-05-04T12:17:23.612 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000303948 s, 1.7 MB/s 2024-05-04T12:17:23.614 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-04T12:17:23.663 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_4 2024-05-04T12:17:23.710 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-04T12:17:23.710 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T12:17:23.710 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 926 Links: 1 2024-05-04T12:17:23.710 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T12:17:23.710 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-05-04 12:11:13.642251170 +0000 2024-05-04T12:17:23.710 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-05-04 12:11:13.370253599 +0000 2024-05-04T12:17:23.710 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-05-04 12:11:13.370253599 +0000 2024-05-04T12:17:23.710 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: - 2024-05-04T12:17:23.710 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-04T12:17:23.763 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-05-04T12:17:23.763 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-05-04T12:17:23.763 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000203389 s, 2.5 MB/s 2024-05-04T12:17:23.764 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-04T12:17:23.811 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-04T12:17:23.811 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:23.811 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2024-05-04T12:17:23.817 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-04T12:17:23.817 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2024-05-04T12:17:23.862 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-04T12:17:23.862 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T12:17:23.863 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 876 Links: 1 2024-05-04T12:17:23.863 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T12:17:23.863 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-05-04 12:11:11.599401510 +0000 2024-05-04T12:17:23.863 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-05-04 12:11:11.343403797 +0000 2024-05-04T12:17:23.863 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-05-04 12:11:11.343403797 +0000 2024-05-04T12:17:23.863 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: - 2024-05-04T12:17:23.863 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-04T12:17:23.917 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-05-04T12:17:23.917 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-05-04T12:17:23.917 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000423919 s, 1.2 MB/s 2024-05-04T12:17:23.918 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-04T12:17:23.967 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2024-05-04T12:17:24.014 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-04T12:17:24.014 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T12:17:24.014 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 892 Links: 1 2024-05-04T12:17:24.014 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T12:17:24.014 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-05-04 12:11:12.091397116 +0000 2024-05-04T12:17:24.014 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-05-04 12:11:11.835399402 +0000 2024-05-04T12:17:24.014 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-05-04 12:11:11.835399402 +0000 2024-05-04T12:17:24.014 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: - 2024-05-04T12:17:24.014 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-04T12:17:24.068 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-05-04T12:17:24.068 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-05-04T12:17:24.068 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000426889 s, 1.2 MB/s 2024-05-04T12:17:24.069 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-04T12:17:24.119 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2024-05-04T12:17:24.166 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-04T12:17:24.166 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T12:17:24.166 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 909 Links: 1 2024-05-04T12:17:24.166 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T12:17:24.166 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-05-04 12:11:12.595392614 +0000 2024-05-04T12:17:24.166 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-05-04 12:11:12.335394937 +0000 2024-05-04T12:17:24.166 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-05-04 12:11:12.335394937 +0000 2024-05-04T12:17:24.166 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: - 2024-05-04T12:17:24.167 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-04T12:17:24.219 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-05-04T12:17:24.220 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-05-04T12:17:24.220 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000323874 s, 1.6 MB/s 2024-05-04T12:17:24.221 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-04T12:17:24.267 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2024-05-04T12:17:24.313 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-04T12:17:24.314 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T12:17:24.314 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 921 Links: 1 2024-05-04T12:17:24.314 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T12:17:24.314 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-05-04 12:11:13.075388326 +0000 2024-05-04T12:17:24.314 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-05-04 12:11:12.827390542 +0000 2024-05-04T12:17:24.314 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-05-04 12:11:12.827390542 +0000 2024-05-04T12:17:24.314 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: - 2024-05-04T12:17:24.314 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-04T12:17:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-05-04T12:17:24.368 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-05-04T12:17:24.368 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000285738 s, 1.8 MB/s 2024-05-04T12:17:24.369 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-04T12:17:24.415 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-04T12:17:24.415 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi052.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi111.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-04T12:17:24.415 INFO:tasks.ceph:Generating config... 2024-05-04T12:17:24.416 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-04T12:17:24.416 INFO:tasks.ceph:[client] debug client = 20 2024-05-04T12:17:24.416 INFO:tasks.ceph:[client] debug ms = 1 2024-05-04T12:17:24.417 INFO:tasks.ceph:[client] pid file = 2024-05-04T12:17:24.417 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-04T12:17:24.417 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-04T12:17:24.417 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-04T12:17:24.417 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-04T12:17:24.417 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-04T12:17:24.417 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-04T12:17:24.417 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-04T12:17:24.417 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-04T12:17:24.417 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-04T12:17:24.417 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-04T12:17:24.417 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-04T12:17:24.417 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-04T12:17:24.417 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-04T12:17:24.418 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-04T12:17:24.418 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-04T12:17:24.418 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-04T12:17:24.418 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-04T12:17:24.418 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-04T12:17:24.418 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-04T12:17:24.418 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-04T12:17:24.418 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-04T12:17:24.418 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-04T12:17:24.418 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-04T12:17:24.418 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-04T12:17:24.418 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-04T12:17:24.418 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-04T12:17:24.418 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-04T12:17:24.418 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-04T12:17:24.419 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-04T12:17:24.419 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-04T12:17:24.419 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-04T12:17:24.419 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-04T12:17:24.419 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-04T12:17:24.419 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-04T12:17:24.419 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-04T12:17:24.419 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-04T12:17:24.419 INFO:tasks.ceph:Setting up mon.a... 2024-05-04T12:17:24.419 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-04T12:17:24.455 INFO:teuthology.orchestra.run.smithi052.stdout:creating /etc/ceph/ceph.keyring 2024-05-04T12:17:24.468 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-04T12:17:24.544 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-04T12:17:24.597 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.52'), ('mon.b', '172.21.15.111'), ('mon.c', '[v2:172.21.15.111:3301,v1:172.21.15.111:6790]')] 2024-05-04T12:17:24.597 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.52,172.21.15.111,[v2:172.21.15.111:3301,v1:172.21.15.111:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-04T12:17:24.598 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:24.598 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-04T12:17:24.645 DEBUG:teuthology.orchestra.run.smithi052:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.52 --add b 172.21.15.111 --addv c '[v2:172.21.15.111:3301,v1:172.21.15.111:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-04T12:17:24.732 INFO:teuthology.orchestra.run.smithi052.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-04T12:17:24.732 INFO:teuthology.orchestra.run.smithi052.stdout:monmaptool: generated fsid 708d9177-9f3e-4c4b-9d29-7b98768b2a25 2024-05-04T12:17:24.732 INFO:teuthology.orchestra.run.smithi052.stdout:setting min_mon_release = quincy 2024-05-04T12:17:24.732 INFO:teuthology.orchestra.run.smithi052.stdout:epoch 0 2024-05-04T12:17:24.732 INFO:teuthology.orchestra.run.smithi052.stdout:fsid 708d9177-9f3e-4c4b-9d29-7b98768b2a25 2024-05-04T12:17:24.732 INFO:teuthology.orchestra.run.smithi052.stdout:last_changed 2024-05-04T12:17:24.735371+0000 2024-05-04T12:17:24.733 INFO:teuthology.orchestra.run.smithi052.stdout:created 2024-05-04T12:17:24.735371+0000 2024-05-04T12:17:24.733 INFO:teuthology.orchestra.run.smithi052.stdout:min_mon_release 17 (quincy) 2024-05-04T12:17:24.733 INFO:teuthology.orchestra.run.smithi052.stdout:election_strategy: 1 2024-05-04T12:17:24.733 INFO:teuthology.orchestra.run.smithi052.stdout:0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-05-04T12:17:24.733 INFO:teuthology.orchestra.run.smithi052.stdout:1: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-05-04T12:17:24.733 INFO:teuthology.orchestra.run.smithi052.stdout:2: [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] mon.c 2024-05-04T12:17:24.733 INFO:teuthology.orchestra.run.smithi052.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-04T12:17:24.734 DEBUG:teuthology.orchestra.run.smithi052:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-04T12:17:24.782 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 708d9177-9f3e-4c4b-9d29-7b98768b2a25... 2024-05-04T12:17:24.783 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-04T12:17:24.827 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-04T12:17:24.856 INFO:teuthology.orchestra.run.smithi052.stdout:[global] 2024-05-04T12:17:24.856 INFO:teuthology.orchestra.run.smithi052.stdout: chdir = "" 2024-05-04T12:17:24.856 INFO:teuthology.orchestra.run.smithi052.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-04T12:17:24.856 INFO:teuthology.orchestra.run.smithi052.stdout: auth supported = cephx 2024-05-04T12:17:24.856 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.856 INFO:teuthology.orchestra.run.smithi052.stdout: filestore xattr use omap = true 2024-05-04T12:17:24.856 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.856 INFO:teuthology.orchestra.run.smithi052.stdout: mon clock drift allowed = 1.000 2024-05-04T12:17:24.856 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.856 INFO:teuthology.orchestra.run.smithi052.stdout: osd crush chooseleaf type = 0 2024-05-04T12:17:24.856 INFO:teuthology.orchestra.run.smithi052.stdout: auth debug = true 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: ms die on old message = true 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: ms die on bug = true 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: mon pg warn max object skew = 0 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: # disable pg_autoscaler by default for new pools 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: osd pool default size = 2 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd allow primary affinity = true 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd allow pg remap = true 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on legacy crush tunables = false 2024-05-04T12:17:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on crush straw calc version zero = false 2024-05-04T12:17:24.858 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on no sortbitwise = false 2024-05-04T12:17:24.858 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on osd down out interval zero = false 2024-05-04T12:17:24.858 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on too few osds = false 2024-05-04T12:17:24.858 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-04T12:17:24.858 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-04T12:17:24.858 INFO:teuthology.orchestra.run.smithi052.stdout: mon_allow_pool_size_one = true 2024-05-04T12:17:24.858 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.858 INFO:teuthology.orchestra.run.smithi052.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-04T12:17:24.858 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.858 INFO:teuthology.orchestra.run.smithi052.stdout: osd default data pool replay window = 5 2024-05-04T12:17:24.858 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.858 INFO:teuthology.orchestra.run.smithi052.stdout: mon allow pool delete = true 2024-05-04T12:17:24.858 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.859 INFO:teuthology.orchestra.run.smithi052.stdout: mon cluster log file level = debug 2024-05-04T12:17:24.859 INFO:teuthology.orchestra.run.smithi052.stdout: debug asserts on shutdown = true 2024-05-04T12:17:24.862 INFO:teuthology.orchestra.run.smithi052.stdout: mon health detail to clog = false 2024-05-04T12:17:24.862 INFO:teuthology.orchestra.run.smithi052.stdout: mon host = "172.21.15.52,172.21.15.111,[v2:172.21.15.111:3301,v1:172.21.15.111:6790]" 2024-05-04T12:17:24.862 INFO:teuthology.orchestra.run.smithi052.stdout: fsid = 708d9177-9f3e-4c4b-9d29-7b98768b2a25 2024-05-04T12:17:24.862 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.862 INFO:teuthology.orchestra.run.smithi052.stdout:[osd] 2024-05-04T12:17:24.862 INFO:teuthology.orchestra.run.smithi052.stdout: osd journal size = 100 2024-05-04T12:17:24.862 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.862 INFO:teuthology.orchestra.run.smithi052.stdout: osd scrub load threshold = 5.0 2024-05-04T12:17:24.862 INFO:teuthology.orchestra.run.smithi052.stdout: osd scrub max interval = 600 2024-05-04T12:17:24.862 INFO:teuthology.orchestra.run.smithi052.stdout: osd mclock profile = high_recovery_ops 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: osd recover clone overlap = true 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: osd recovery max chunk = 1048576 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug shutdown = true 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug op order = true 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug verify stray on activate = true 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug trim objects = true 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: osd open classes on start = true 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug pg log writeout = true 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: osd deep scrub update digest min age = 30 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: osd map max advance = 10 2024-05-04T12:17:24.863 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: journal zero on create = true 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: filestore ondisk finisher threads = 3 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: filestore apply finisher threads = 3 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: bdev debug aio = true 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug misdirected ops = true 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: bdev async discard = True 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: bdev enable discard = True 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: bluestore allocator = bitmap 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: bluestore block size = 96636764160 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: bluestore fsck on mount = True 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: debug bluefs = 1/20 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: debug bluestore = 1/20 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-05-04T12:17:24.864 INFO:teuthology.orchestra.run.smithi052.stdout: debug osd = 20 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: debug rocksdb = 4/10 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd backfillfull_ratio = 0.85 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd full ratio = 0.9 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd nearfull ratio = 0.8 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: osd failsafe full ratio = 0.95 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: osd objectstore = bluestore 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: osd op complaint time = 180 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout:[mgr] 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: debug mgr = 20 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: debug mon = 20 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: debug auth = 20 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min pgs per osd = 4 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min bytes per osd = 10 2024-05-04T12:17:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: mgr/telemetry/nag = false 2024-05-04T12:17:24.866 INFO:teuthology.orchestra.run.smithi052.stdout: debug client = 20 2024-05-04T12:17:24.866 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.866 INFO:teuthology.orchestra.run.smithi052.stdout:[mon] 2024-05-04T12:17:24.866 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-05-04T12:17:24.866 INFO:teuthology.orchestra.run.smithi052.stdout: debug mon = 20 2024-05-04T12:17:24.866 INFO:teuthology.orchestra.run.smithi052.stdout: debug paxos = 20 2024-05-04T12:17:24.866 INFO:teuthology.orchestra.run.smithi052.stdout: debug auth = 20 2024-05-04T12:17:24.866 INFO:teuthology.orchestra.run.smithi052.stdout: mon data avail warn = 5 2024-05-04T12:17:24.866 INFO:teuthology.orchestra.run.smithi052.stdout: mon mgr mkfs grace = 240 2024-05-04T12:17:24.866 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min pgs per osd = 4 2024-05-04T12:17:24.866 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd reporter subtree level = osd 2024-05-04T12:17:24.866 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd prime pg temp = true 2024-05-04T12:17:24.866 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min bytes per osd = 10 2024-05-04T12:17:24.866 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.867 INFO:teuthology.orchestra.run.smithi052.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-04T12:17:24.867 INFO:teuthology.orchestra.run.smithi052.stdout: auth mon ticket ttl = 660 # 11m 2024-05-04T12:17:24.867 INFO:teuthology.orchestra.run.smithi052.stdout: auth service ticket ttl = 240 # 4m 2024-05-04T12:17:24.867 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.867 INFO:teuthology.orchestra.run.smithi052.stdout: # don't complain about insecure global_id in the test suite 2024-05-04T12:17:24.867 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-04T12:17:24.867 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-04T12:17:24.867 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.867 INFO:teuthology.orchestra.run.smithi052.stdout: # 1m isn't quite enough 2024-05-04T12:17:24.867 INFO:teuthology.orchestra.run.smithi052.stdout: mon_down_mkfs_grace = 2m 2024-05-04T12:17:24.867 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.867 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_filestore_osds = false 2024-05-04T12:17:24.867 INFO:teuthology.orchestra.run.smithi052.stdout: mon down mkfs grace = 300 2024-05-04T12:17:24.867 INFO:teuthology.orchestra.run.smithi052.stdout: mon op complaint time = 120 2024-05-04T12:17:24.868 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:17:24.868 INFO:teuthology.orchestra.run.smithi052.stdout:[client] 2024-05-04T12:17:24.868 INFO:teuthology.orchestra.run.smithi052.stdout: rgw cache enabled = true 2024-05-04T12:17:24.868 INFO:teuthology.orchestra.run.smithi052.stdout: rgw enable ops log = true 2024-05-04T12:17:24.868 INFO:teuthology.orchestra.run.smithi052.stdout: rgw enable usage log = true 2024-05-04T12:17:24.868 INFO:teuthology.orchestra.run.smithi052.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-04T12:17:24.868 INFO:teuthology.orchestra.run.smithi052.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-04T12:17:24.868 INFO:teuthology.orchestra.run.smithi052.stdout: client mount timeout = 600 2024-05-04T12:17:24.868 INFO:teuthology.orchestra.run.smithi052.stdout: debug client = 20 2024-05-04T12:17:24.868 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-05-04T12:17:24.868 INFO:teuthology.orchestra.run.smithi052.stdout: pid file = "" 2024-05-04T12:17:24.868 INFO:teuthology.orchestra.run.smithi052.stdout: rados mon op timeout = 900 2024-05-04T12:17:24.868 INFO:teuthology.orchestra.run.smithi052.stdout: rados osd op timeout = 900 2024-05-04T12:17:24.868 INFO:teuthology.orchestra.run.smithi052.stdout:[mon.a] 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout:[mon.b] 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout:[mon.c] 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.a] 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.c] 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.b] 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.d] 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout:[mds] 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout: debug mds = 20 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout: debug mds balancer = 20 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout: mds debug frag = True 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout: mds debug scatterstat = True 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout: mds op complaint time = 180 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout: mds verify scatter = True 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout: osd op complaint time = 180 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout: rados mon op timeout = 900 2024-05-04T12:17:24.869 INFO:teuthology.orchestra.run.smithi052.stdout: rados osd op timeout = 900 2024-05-04T12:17:24.871 INFO:teuthology.orchestra.run.smithi111.stdout:[global] 2024-05-04T12:17:24.871 INFO:teuthology.orchestra.run.smithi111.stdout: chdir = "" 2024-05-04T12:17:24.871 INFO:teuthology.orchestra.run.smithi111.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-04T12:17:24.871 INFO:teuthology.orchestra.run.smithi111.stdout: auth supported = cephx 2024-05-04T12:17:24.871 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.871 INFO:teuthology.orchestra.run.smithi111.stdout: filestore xattr use omap = true 2024-05-04T12:17:24.871 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: mon clock drift allowed = 1.000 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: osd crush chooseleaf type = 0 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: auth debug = true 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on old message = true 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on bug = true 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: mon pg warn max object skew = 0 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: # disable pg_autoscaler by default for new pools 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: osd pool default size = 2 2024-05-04T12:17:24.872 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd allow primary affinity = true 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd allow pg remap = true 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on legacy crush tunables = false 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on crush straw calc version zero = false 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on no sortbitwise = false 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on osd down out interval zero = false 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on too few osds = false 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: mon_allow_pool_size_one = true 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: osd default data pool replay window = 5 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: mon allow pool delete = true 2024-05-04T12:17:24.873 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.874 INFO:teuthology.orchestra.run.smithi111.stdout: mon cluster log file level = debug 2024-05-04T12:17:24.874 INFO:teuthology.orchestra.run.smithi111.stdout: debug asserts on shutdown = true 2024-05-04T12:17:24.874 INFO:teuthology.orchestra.run.smithi111.stdout: mon health detail to clog = false 2024-05-04T12:17:24.874 INFO:teuthology.orchestra.run.smithi111.stdout: mon host = "172.21.15.52,172.21.15.111,[v2:172.21.15.111:3301,v1:172.21.15.111:6790]" 2024-05-04T12:17:24.874 INFO:teuthology.orchestra.run.smithi111.stdout: fsid = 708d9177-9f3e-4c4b-9d29-7b98768b2a25 2024-05-04T12:17:24.874 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.874 INFO:teuthology.orchestra.run.smithi111.stdout:[osd] 2024-05-04T12:17:24.874 INFO:teuthology.orchestra.run.smithi111.stdout: osd journal size = 100 2024-05-04T12:17:24.874 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.874 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub load threshold = 5.0 2024-05-04T12:17:24.874 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub max interval = 600 2024-05-04T12:17:24.874 INFO:teuthology.orchestra.run.smithi111.stdout: osd mclock profile = high_recovery_ops 2024-05-04T12:17:24.874 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: osd recover clone overlap = true 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: osd recovery max chunk = 1048576 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug shutdown = true 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug op order = true 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug verify stray on activate = true 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug trim objects = true 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: osd open classes on start = true 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug pg log writeout = true 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: osd deep scrub update digest min age = 30 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: osd map max advance = 10 2024-05-04T12:17:24.875 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.876 INFO:teuthology.orchestra.run.smithi111.stdout: journal zero on create = true 2024-05-04T12:17:24.876 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.876 INFO:teuthology.orchestra.run.smithi111.stdout: filestore ondisk finisher threads = 3 2024-05-04T12:17:24.876 INFO:teuthology.orchestra.run.smithi111.stdout: filestore apply finisher threads = 3 2024-05-04T12:17:24.876 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.876 INFO:teuthology.orchestra.run.smithi111.stdout: bdev debug aio = true 2024-05-04T12:17:24.876 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug misdirected ops = true 2024-05-04T12:17:24.876 INFO:teuthology.orchestra.run.smithi111.stdout: bdev async discard = True 2024-05-04T12:17:24.876 INFO:teuthology.orchestra.run.smithi111.stdout: bdev enable discard = True 2024-05-04T12:17:24.876 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore allocator = bitmap 2024-05-04T12:17:24.876 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore block size = 96636764160 2024-05-04T12:17:24.876 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore fsck on mount = True 2024-05-04T12:17:24.876 INFO:teuthology.orchestra.run.smithi111.stdout: debug bluefs = 1/20 2024-05-04T12:17:24.876 INFO:teuthology.orchestra.run.smithi111.stdout: debug bluestore = 1/20 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout: debug osd = 20 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout: debug rocksdb = 4/10 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd backfillfull_ratio = 0.85 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd full ratio = 0.9 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd nearfull ratio = 0.8 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout: osd failsafe full ratio = 0.95 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout: osd objectstore = bluestore 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout: osd op complaint time = 180 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr] 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout: debug mgr = 20 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout: debug mon = 20 2024-05-04T12:17:24.877 INFO:teuthology.orchestra.run.smithi111.stdout: debug auth = 20 2024-05-04T12:17:24.878 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min pgs per osd = 4 2024-05-04T12:17:24.878 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min bytes per osd = 10 2024-05-04T12:17:24.878 INFO:teuthology.orchestra.run.smithi111.stdout: mgr/telemetry/nag = false 2024-05-04T12:17:24.878 INFO:teuthology.orchestra.run.smithi111.stdout: debug client = 20 2024-05-04T12:17:24.878 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.878 INFO:teuthology.orchestra.run.smithi111.stdout:[mon] 2024-05-04T12:17:24.878 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-05-04T12:17:24.878 INFO:teuthology.orchestra.run.smithi111.stdout: debug mon = 20 2024-05-04T12:17:24.878 INFO:teuthology.orchestra.run.smithi111.stdout: debug paxos = 20 2024-05-04T12:17:24.878 INFO:teuthology.orchestra.run.smithi111.stdout: debug auth = 20 2024-05-04T12:17:24.878 INFO:teuthology.orchestra.run.smithi111.stdout: mon data avail warn = 5 2024-05-04T12:17:24.878 INFO:teuthology.orchestra.run.smithi111.stdout: mon mgr mkfs grace = 240 2024-05-04T12:17:24.878 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min pgs per osd = 4 2024-05-04T12:17:24.878 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd reporter subtree level = osd 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd prime pg temp = true 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min bytes per osd = 10 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: auth mon ticket ttl = 660 # 11m 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: auth service ticket ttl = 240 # 4m 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: # don't complain about insecure global_id in the test suite 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: # 1m isn't quite enough 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: mon_down_mkfs_grace = 2m 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_filestore_osds = false 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: mon down mkfs grace = 300 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: mon op complaint time = 120 2024-05-04T12:17:24.879 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout:[client] 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout: rgw cache enabled = true 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout: rgw enable ops log = true 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout: rgw enable usage log = true 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout: client mount timeout = 600 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout: debug client = 20 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout: pid file = "" 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout: rados mon op timeout = 900 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout: rados osd op timeout = 900 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.a] 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.b] 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.c] 2024-05-04T12:17:24.880 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.a] 2024-05-04T12:17:24.881 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.c] 2024-05-04T12:17:24.881 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.b] 2024-05-04T12:17:24.881 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.d] 2024-05-04T12:17:24.881 INFO:teuthology.orchestra.run.smithi111.stdout:[mds] 2024-05-04T12:17:24.881 INFO:teuthology.orchestra.run.smithi111.stdout: debug mds = 20 2024-05-04T12:17:24.881 INFO:teuthology.orchestra.run.smithi111.stdout: debug mds balancer = 20 2024-05-04T12:17:24.881 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-05-04T12:17:24.881 INFO:teuthology.orchestra.run.smithi111.stdout: mds debug frag = True 2024-05-04T12:17:24.881 INFO:teuthology.orchestra.run.smithi111.stdout: mds debug scatterstat = True 2024-05-04T12:17:24.881 INFO:teuthology.orchestra.run.smithi111.stdout: mds op complaint time = 180 2024-05-04T12:17:24.881 INFO:teuthology.orchestra.run.smithi111.stdout: mds verify scatter = True 2024-05-04T12:17:24.881 INFO:teuthology.orchestra.run.smithi111.stdout: osd op complaint time = 180 2024-05-04T12:17:24.881 INFO:teuthology.orchestra.run.smithi111.stdout: rados mon op timeout = 900 2024-05-04T12:17:24.881 INFO:teuthology.orchestra.run.smithi111.stdout: rados osd op timeout = 900 2024-05-04T12:17:24.882 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-04T12:17:24.882 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-04T12:17:24.963 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-04T12:17:24.963 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:24.963 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-04T12:17:24.968 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:24.968 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-04T12:17:25.014 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T12:17:25.014 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:25.014 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-04T12:17:25.014 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-04T12:17:25.079 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:25.079 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-04T12:17:25.129 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T12:17:25.130 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:25.130 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-04T12:17:25.130 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-04T12:17:25.154 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:25.154 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-04T12:17:25.202 INFO:tasks.ceph:Setting up mon nodes... 2024-05-04T12:17:25.202 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-04T12:17:25.202 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-04T12:17:25.248 INFO:teuthology.orchestra.run.smithi052.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-04T12:17:25.252 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-04T12:17:25.295 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-04T12:17:25.329 INFO:tasks.ceph:Setting up mds nodes... 2024-05-04T12:17:25.329 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-04T12:17:25.373 INFO:teuthology.orchestra.run.smithi052.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-04T12:17:25.376 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-04T12:17:25.389 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-04T12:17:25.478 INFO:teuthology.orchestra.run.smithi052.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-04T12:17:25.481 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-04T12:17:25.537 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-04T12:17:25.583 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-04T12:17:25.586 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-04T12:17:25.598 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-04T12:17:25.679 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-04T12:17:25.682 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-04T12:17:25.738 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-04T12:17:25.738 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-04T12:17:25.775 INFO:teuthology.orchestra.run.smithi052.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-04T12:17:25.788 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-04T12:17:25.862 INFO:teuthology.orchestra.run.smithi052.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-04T12:17:25.873 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-04T12:17:25.909 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-04T12:17:25.921 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-04T12:17:25.998 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-04T12:17:26.009 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-04T12:17:26.009 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi052.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi111.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-04T12:17:26.009 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-04T12:17:26.023 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-04T12:17:26.024 INFO:tasks.ceph:role: osd.0 2024-05-04T12:17:26.024 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T12:17:26.024 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-04T12:17:26.078 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-04T12:17:26.079 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-04T12:17:26.079 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-04T12:17:26.079 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-04T12:17:26.079 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-04T12:17:26.079 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-05-04T12:17:26.079 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-04T12:17:26.079 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-04T12:17:26.079 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-04T12:17:26.079 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-04T12:17:26.102 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-05-04T12:17:26.112 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-05-04T12:17:26.112 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-04T12:17:26.167 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-04T12:17:26.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T12:17:26.218 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: /sbin/restorecon: command not found 2024-05-04T12:17:26.218 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-04T12:17:26.270 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-04T12:17:26.271 INFO:tasks.ceph:role: osd.1 2024-05-04T12:17:26.271 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T12:17:26.271 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-04T12:17:26.325 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-04T12:17:26.326 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-04T12:17:26.326 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-04T12:17:26.326 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-04T12:17:26.326 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-04T12:17:26.326 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-05-04T12:17:26.326 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-04T12:17:26.326 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-04T12:17:26.326 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-04T12:17:26.326 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-04T12:17:26.349 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-05-04T12:17:26.350 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-05-04T12:17:26.351 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-04T12:17:26.407 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-04T12:17:26.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T12:17:26.459 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: /sbin/restorecon: command not found 2024-05-04T12:17:26.459 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-04T12:17:26.517 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-04T12:17:26.517 INFO:tasks.ceph:role: osd.2 2024-05-04T12:17:26.517 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T12:17:26.517 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-04T12:17:26.573 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-04T12:17:26.573 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-04T12:17:26.573 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-04T12:17:26.573 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-04T12:17:26.573 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-04T12:17:26.573 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-05-04T12:17:26.573 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-04T12:17:26.573 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-04T12:17:26.573 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-04T12:17:26.573 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-04T12:17:26.596 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-05-04T12:17:26.598 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-05-04T12:17:26.598 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-04T12:17:26.657 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-04T12:17:26.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T12:17:26.711 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: /sbin/restorecon: command not found 2024-05-04T12:17:26.711 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-04T12:17:26.763 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-04T12:17:26.763 INFO:tasks.ceph:role: osd.3 2024-05-04T12:17:26.763 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T12:17:26.764 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-04T12:17:26.817 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-04T12:17:26.817 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-04T12:17:26.818 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-04T12:17:26.818 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-04T12:17:26.818 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-04T12:17:26.818 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-05-04T12:17:26.818 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-04T12:17:26.818 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-04T12:17:26.818 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-04T12:17:26.818 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-04T12:17:26.841 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-05-04T12:17:26.843 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-05-04T12:17:26.843 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-04T12:17:26.899 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-04T12:17:26.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T12:17:26.950 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: /sbin/restorecon: command not found 2024-05-04T12:17:26.951 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-04T12:17:27.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:27.152+0000 7f67cf04e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-04T12:17:27.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:27.152+0000 7f67cf04e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-04T12:17:27.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:27.152+0000 7f67cf04e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-04T12:17:27.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:27.152+0000 7f67cf04e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-04T12:17:27.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:27.152+0000 7f67cf04e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-04T12:17:27.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:27.152+0000 7f67cf04e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-04T12:17:30.156 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-04T12:17:30.170 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-04T12:17:30.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:30.256+0000 7f3abe4957c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-04T12:17:30.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:30.256+0000 7f3abe4957c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-04T12:17:30.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:30.256+0000 7f3abe4957c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-04T12:17:30.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:30.256+0000 7f3abe4957c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-04T12:17:30.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:30.256+0000 7f3abe4957c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-04T12:17:30.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:30.256+0000 7f3abe4957c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-04T12:17:33.221 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-04T12:17:33.279 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-04T12:17:33.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:33.360+0000 7f07f18337c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-04T12:17:33.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:33.364+0000 7f07f18337c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-04T12:17:33.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:33.364+0000 7f07f18337c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-04T12:17:33.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:33.364+0000 7f07f18337c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-04T12:17:33.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:33.364+0000 7f07f18337c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-04T12:17:33.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:33.364+0000 7f07f18337c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-04T12:17:36.328 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-04T12:17:36.383 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-04T12:17:36.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:36.464+0000 7f4fbad877c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-04T12:17:36.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:36.464+0000 7f4fbad877c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-04T12:17:36.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:36.468+0000 7f4fbad877c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-04T12:17:36.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:36.468+0000 7f4fbad877c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-04T12:17:36.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:36.468+0000 7f4fbad877c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-04T12:17:36.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:36.468+0000 7f4fbad877c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-04T12:17:39.456 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-04T12:17:39.510 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-04T12:17:39.525 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-04T12:17:39.525 INFO:tasks.ceph:role: osd.4 2024-05-04T12:17:39.525 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T12:17:39.525 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-04T12:17:39.590 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-04T12:17:39.590 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-04T12:17:39.591 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-04T12:17:39.591 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-04T12:17:39.591 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-04T12:17:39.591 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-05-04T12:17:39.591 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-04T12:17:39.591 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-04T12:17:39.591 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-04T12:17:39.591 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-04T12:17:40.040 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-05-04T12:17:40.043 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-05-04T12:17:40.043 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-04T12:17:40.143 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-04T12:17:40.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T12:17:40.154 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: /sbin/restorecon: command not found 2024-05-04T12:17:40.155 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-04T12:17:40.210 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-04T12:17:40.210 INFO:tasks.ceph:role: osd.5 2024-05-04T12:17:40.210 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T12:17:40.210 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-04T12:17:40.274 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-04T12:17:40.274 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-04T12:17:40.274 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-04T12:17:40.274 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-04T12:17:40.274 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-04T12:17:40.274 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-05-04T12:17:40.274 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-04T12:17:40.275 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-04T12:17:40.275 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-04T12:17:40.275 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-04T12:17:40.736 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-05-04T12:17:40.738 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-05-04T12:17:40.738 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-04T12:17:40.899 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-04T12:17:40.910 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: /sbin/restorecon: command not found 2024-05-04T12:17:40.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T12:17:40.911 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-04T12:17:40.966 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-04T12:17:40.966 INFO:tasks.ceph:role: osd.6 2024-05-04T12:17:40.966 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T12:17:40.966 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-04T12:17:41.030 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-04T12:17:41.030 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-04T12:17:41.030 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-04T12:17:41.030 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-04T12:17:41.031 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-04T12:17:41.031 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-05-04T12:17:41.031 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-04T12:17:41.031 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-04T12:17:41.031 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-04T12:17:41.031 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-04T12:17:41.503 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-05-04T12:17:41.505 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-05-04T12:17:41.505 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-04T12:17:41.658 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-04T12:17:41.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T12:17:41.670 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: /sbin/restorecon: command not found 2024-05-04T12:17:41.670 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-04T12:17:41.725 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-04T12:17:41.726 INFO:tasks.ceph:role: osd.7 2024-05-04T12:17:41.726 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T12:17:41.726 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-04T12:17:41.790 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-04T12:17:41.791 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-04T12:17:41.791 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-04T12:17:41.791 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-04T12:17:41.791 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-04T12:17:41.791 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-05-04T12:17:41.791 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-04T12:17:41.791 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-04T12:17:41.791 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-04T12:17:41.791 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-04T12:17:42.211 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-05-04T12:17:42.213 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-05-04T12:17:42.213 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-04T12:17:42.299 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-04T12:17:42.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T12:17:42.309 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: /sbin/restorecon: command not found 2024-05-04T12:17:42.309 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-04T12:17:42.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:42.419+0000 7f32c85107c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-04T12:17:42.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:42.423+0000 7f32c85107c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-04T12:17:42.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:42.423+0000 7f32c85107c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-04T12:17:42.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:42.423+0000 7f32c85107c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-04T12:17:42.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:42.423+0000 7f32c85107c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-04T12:17:42.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:42.431+0000 7f32c85107c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-04T12:17:45.433 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-04T12:17:45.447 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-04T12:17:45.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:45.523+0000 7f69b31c17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-04T12:17:45.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:45.523+0000 7f69b31c17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-04T12:17:45.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:45.523+0000 7f69b31c17c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-04T12:17:45.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:45.523+0000 7f69b31c17c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-04T12:17:45.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:45.523+0000 7f69b31c17c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-04T12:17:45.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:45.523+0000 7f69b31c17c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-04T12:17:48.504 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-04T12:17:48.562 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-04T12:17:48.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:48.639+0000 7f7b00aef7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-04T12:17:48.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:48.639+0000 7f7b00aef7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-04T12:17:48.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:48.639+0000 7f7b00aef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-04T12:17:48.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:48.639+0000 7f7b00aef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-04T12:17:48.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:48.639+0000 7f7b00aef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-04T12:17:48.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:48.639+0000 7f7b00aef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-04T12:17:51.630 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-04T12:17:51.686 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-04T12:17:51.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:51.759+0000 7f2999e0b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-04T12:17:51.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:51.759+0000 7f2999e0b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-04T12:17:51.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:51.759+0000 7f2999e0b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-04T12:17:51.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:51.759+0000 7f2999e0b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-04T12:17:51.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:51.759+0000 7f2999e0b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-04T12:17:51.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:51.759+0000 7f2999e0b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-04T12:17:53.091 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:17:53.094 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:17:54.778 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-04T12:17:54.791 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-04T12:17:54.791 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:54.791 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-04T12:17:54.806 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:54.806 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-04T12:17:54.861 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:54.861 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-04T12:17:54.917 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:54.917 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-04T12:17:54.974 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:54.974 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-04T12:17:55.029 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:55.029 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-04T12:17:55.086 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:55.086 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-04T12:17:55.141 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:55.141 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-04T12:17:55.154 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:55.155 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-04T12:17:55.208 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:55.209 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-04T12:17:55.261 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:55.261 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-04T12:17:55.317 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:55.317 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-04T12:17:55.373 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:55.373 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-04T12:17:55.429 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:55.429 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-04T12:17:55.486 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:55.486 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-04T12:17:55.492 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:55.492 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-04T12:17:55.538 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:55.539 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-04T12:17:55.544 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:55.544 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-04T12:17:55.590 INFO:tasks.ceph:Adding keys to all mons... 2024-05-04T12:17:55.590 DEBUG:teuthology.orchestra.run.smithi052:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-04T12:17:55.593 DEBUG:teuthology.orchestra.run.smithi111:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-04T12:17:55.640 INFO:teuthology.orchestra.run.smithi052.stdout:[mgr.y] 2024-05-04T12:17:55.640 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBVJzZmXy4GDxAA6TDE57YwmNNvp2Wmkni8OQ== 2024-05-04T12:17:55.640 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.a] 2024-05-04T12:17:55.640 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBVJzZmLE53FhAAbGwz3qGhQTZmf2M7JAzqLQ== 2024-05-04T12:17:55.640 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.c] 2024-05-04T12:17:55.640 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBVJzZm9ue4HBAAyNkrNG+mHYXsiFqsZg+WTA== 2024-05-04T12:17:55.640 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.0] 2024-05-04T12:17:55.640 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBXJzZmR7w9CRAAk3eqOAw/hgbLNGgR306s8g== 2024-05-04T12:17:55.640 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.1] 2024-05-04T12:17:55.640 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBaJzZmrX1kDxAAvRHfzSkgWpASb8Z09y5Zdw== 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.2] 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBdJzZm6CG1FRAAWkuJidO1JSlt5RTWTf0HAg== 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.3] 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBgJzZmF+TmGxAAla+oLW9K4xDU0n9+tCzYmQ== 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout:[mgr.x] 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBVJzZmRRC8ERAAbWLMj3635LsYGHfICJ1CDQ== 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.b] 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBVJzZmwWHjIhAAHZBv9NPrPIaRLzttDSoPSA== 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.d] 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBVJzZmB9qbKBAAyijx2yI3L0xfXg/JqR6stg== 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.4] 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBmJzZm2fBrGRAAwi+07gy0Pcwnq6WIcREE4Q== 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.5] 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBpJzZmx2WVHxAAyHrMJ70jj7iTXAcme50Lcg== 2024-05-04T12:17:55.641 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.6] 2024-05-04T12:17:55.642 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBsJzZmq9BuJhAAV5wRyq1ERTydpwTQf4r6aw== 2024-05-04T12:17:55.642 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.7] 2024-05-04T12:17:55.642 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBvJzZm2pqcLRAA3S4YRX+XDbvL08IlQ8IuLg== 2024-05-04T12:17:55.642 INFO:teuthology.orchestra.run.smithi052.stdout:[client.0] 2024-05-04T12:17:55.642 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBVJzZm/x1tLhAApncXD+0NjD0NbekN+kVgdQ== 2024-05-04T12:17:55.642 INFO:teuthology.orchestra.run.smithi052.stdout:[client.2] 2024-05-04T12:17:55.642 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBVJzZmMjeYMxAA1T1RD7vNODSe3lVQLsgokA== 2024-05-04T12:17:55.642 INFO:teuthology.orchestra.run.smithi052.stdout:[client.1] 2024-05-04T12:17:55.642 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBVJzZmLPdXNhAAHRdLfb1zoNaeY/9O9UrEmA== 2024-05-04T12:17:55.642 INFO:teuthology.orchestra.run.smithi052.stdout:[client.3] 2024-05-04T12:17:55.642 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBWJzZmTHAFABAAW0rbCV51SnDTWE1drZsK+w== 2024-05-04T12:17:55.648 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr.y] 2024-05-04T12:17:55.648 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBVJzZmXy4GDxAA6TDE57YwmNNvp2Wmkni8OQ== 2024-05-04T12:17:55.648 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.a] 2024-05-04T12:17:55.648 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBVJzZmLE53FhAAbGwz3qGhQTZmf2M7JAzqLQ== 2024-05-04T12:17:55.648 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.c] 2024-05-04T12:17:55.649 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBVJzZm9ue4HBAAyNkrNG+mHYXsiFqsZg+WTA== 2024-05-04T12:17:55.649 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.0] 2024-05-04T12:17:55.649 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBXJzZmR7w9CRAAk3eqOAw/hgbLNGgR306s8g== 2024-05-04T12:17:55.649 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.1] 2024-05-04T12:17:55.649 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBaJzZmrX1kDxAAvRHfzSkgWpASb8Z09y5Zdw== 2024-05-04T12:17:55.649 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.2] 2024-05-04T12:17:55.649 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBdJzZm6CG1FRAAWkuJidO1JSlt5RTWTf0HAg== 2024-05-04T12:17:55.649 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.3] 2024-05-04T12:17:55.649 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBgJzZmF+TmGxAAla+oLW9K4xDU0n9+tCzYmQ== 2024-05-04T12:17:55.649 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr.x] 2024-05-04T12:17:55.649 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBVJzZmRRC8ERAAbWLMj3635LsYGHfICJ1CDQ== 2024-05-04T12:17:55.649 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.b] 2024-05-04T12:17:55.649 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBVJzZmwWHjIhAAHZBv9NPrPIaRLzttDSoPSA== 2024-05-04T12:17:55.649 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.d] 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBVJzZmB9qbKBAAyijx2yI3L0xfXg/JqR6stg== 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.4] 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBmJzZm2fBrGRAAwi+07gy0Pcwnq6WIcREE4Q== 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.5] 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBpJzZmx2WVHxAAyHrMJ70jj7iTXAcme50Lcg== 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.6] 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBsJzZmq9BuJhAAV5wRyq1ERTydpwTQf4r6aw== 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.7] 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBvJzZm2pqcLRAA3S4YRX+XDbvL08IlQ8IuLg== 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout:[client.0] 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBVJzZm/x1tLhAApncXD+0NjD0NbekN+kVgdQ== 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout:[client.2] 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBVJzZmMjeYMxAA1T1RD7vNODSe3lVQLsgokA== 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout:[client.1] 2024-05-04T12:17:55.650 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBVJzZmLPdXNhAAHRdLfb1zoNaeY/9O9UrEmA== 2024-05-04T12:17:55.651 INFO:teuthology.orchestra.run.smithi111.stdout:[client.3] 2024-05-04T12:17:55.651 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBWJzZmTHAFABAAW0rbCV51SnDTWE1drZsK+w== 2024-05-04T12:17:55.651 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-04T12:17:55.687 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-04T12:17:55.738 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-04T12:17:55.740 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-04T12:17:55.826 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-04T12:17:55.829 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-04T12:17:55.873 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:55.919 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:55.963 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:55.965 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:56.039 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:56.051 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:56.119 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:56.135 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:56.199 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-04T12:17:56.219 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-04T12:17:56.278 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-04T12:17:56.303 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-04T12:17:56.362 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-04T12:17:56.365 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-04T12:17:56.449 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:56.452 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:56.531 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:56.535 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:56.613 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:56.619 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:56.700 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:56.703 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-04T12:17:56.784 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-04T12:17:56.787 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-04T12:17:56.869 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-04T12:17:56.872 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-04T12:17:56.956 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-04T12:17:56.958 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-04T12:17:57.041 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-04T12:17:57.043 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-04T12:17:57.126 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-04T12:17:57.127 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-04T12:17:57.140 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-04T12:17:57.233 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-04T12:17:57.244 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-04T12:17:57.256 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-04T12:17:57.348 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-04T12:17:57.360 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-04T12:17:57.411 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-04T12:17:57.501 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-04T12:17:57.511 DEBUG:teuthology.orchestra.run.smithi052:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-04T12:17:57.513 DEBUG:teuthology.orchestra.run.smithi111:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-04T12:17:57.558 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-04T12:17:57.558 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-04T12:17:57.558 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-04T12:17:57.561 INFO:tasks.ceph.mon.a:Started 2024-05-04T12:17:57.561 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-04T12:17:57.561 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-04T12:17:57.603 INFO:tasks.ceph.mon.b:Started 2024-05-04T12:17:57.603 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-04T12:17:57.603 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-04T12:17:57.606 INFO:tasks.ceph.mon.c:Started 2024-05-04T12:17:57.606 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-04T12:17:57.606 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-04T12:17:57.606 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-04T12:17:57.609 INFO:tasks.ceph.mgr.y:Started 2024-05-04T12:17:57.609 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-04T12:17:57.609 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-04T12:17:57.611 INFO:tasks.ceph.mgr.x:Started 2024-05-04T12:17:57.611 INFO:tasks.ceph:Setting crush tunables to default 2024-05-04T12:17:57.612 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --cluster ceph osd crush tunables default 2024-05-04T12:17:57.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4cd65b2640 1 Processor -- start 2024-05-04T12:17:57.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4cd65b2640 1 -- start start 2024-05-04T12:17:57.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4cd65b2640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 0x7f4cd00b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:57.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4cd65b2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4cd0160380 0x7f4cd01607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:57.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4cd65b2640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cd00cd360 con 0x7f4cd00cce10 2024-05-04T12:17:57.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4cd65b2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f4cd00cdc00 con 0x7f4cd0160380 2024-05-04T12:17:57.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4cd65b2640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd00cd640 con 0x7f4cd01531b0 2024-05-04T12:17:57.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4ccffff640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 0x7f4cd00b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:57.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4cd4b4f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4cd0160380 0x7f4cd01607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:57.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4ccffff640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 0x7f4cd00b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.52:51354/0 (socket says 172.21.15.52:51354) 2024-05-04T12:17:57.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4ccffff640 1 -- 172.21.15.52:0/1912267447 learned_addr learned my addr 172.21.15.52:0/1912267447 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:17:57.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4ccffff640 1 -- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 msgr2=0x7f4cd00b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-04T12:17:57.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4ccffff640 1 -- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 msgr2=0x7f4cd00b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-04T12:17:57.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4ccffff640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 0x7f4cd00b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-04T12:17:57.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4cd4b4f640 1 -- 172.21.15.52:0/1912267447 >> v2:172.21.15.111:3300/0 conn(0x7f4cd0160380 msgr2=0x7f4cd01607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-04T12:17:57.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4cd4b4f640 1 -- 172.21.15.52:0/1912267447 >> v2:172.21.15.111:3300/0 conn(0x7f4cd0160380 msgr2=0x7f4cd01607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-04T12:17:57.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4cd4b4f640 1 --2- 172.21.15.52:0/1912267447 >> v2:172.21.15.111:3300/0 conn(0x7f4cd0160380 0x7f4cd01607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-04T12:17:57.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4ccffff640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 0x7f4cd00b4660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T12:17:57.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.727+0000 7f4cd4b4f640 1 --2- 172.21.15.52:0/1912267447 >> v2:172.21.15.111:3300/0 conn(0x7f4cd0160380 0x7f4cd01607f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T12:17:57.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.931+0000 7f4ccffff640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 0x7f4cd00b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.931+0000 7f4cd4b4f640 1 --2- 172.21.15.52:0/1912267447 >> v2:172.21.15.111:3300/0 conn(0x7f4cd0160380 0x7f4cd01607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.931+0000 7f4ccffff640 1 -- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 msgr2=0x7f4cd00b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-04T12:17:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.931+0000 7f4ccffff640 1 -- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 msgr2=0x7f4cd00b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-04T12:17:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.931+0000 7f4ccffff640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 0x7f4cd00b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-04T12:17:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.931+0000 7f4ccffff640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 0x7f4cd00b4660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-04T12:17:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.931+0000 7f4cd4b4f640 1 -- 172.21.15.52:0/1912267447 >> v2:172.21.15.111:3300/0 conn(0x7f4cd0160380 msgr2=0x7f4cd01607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-04T12:17:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.931+0000 7f4cd4b4f640 1 -- 172.21.15.52:0/1912267447 >> v2:172.21.15.111:3300/0 conn(0x7f4cd0160380 msgr2=0x7f4cd01607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-04T12:17:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.931+0000 7f4cd4b4f640 1 --2- 172.21.15.52:0/1912267447 >> v2:172.21.15.111:3300/0 conn(0x7f4cd0160380 0x7f4cd01607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-04T12:17:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.931+0000 7f4cd4b4f640 1 --2- 172.21.15.52:0/1912267447 >> v2:172.21.15.111:3300/0 conn(0x7f4cd0160380 0x7f4cd01607f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-04T12:17:57.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.959+0000 7f4cceffd640 1 -- 172.21.15.52:0/1912267447 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 900873902 0 0) 0x7f4cd00cd360 con 0x7f4cd00cce10 2024-05-04T12:17:57.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.959+0000 7f4cceffd640 1 -- 172.21.15.52:0/1912267447 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cb8003530 con 0x7f4cd00cce10 2024-05-04T12:17:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.959+0000 7f4cceffd640 1 -- 172.21.15.52:0/1912267447 <== mon.1 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 370377121 0 0) 0x7f4cb8003530 con 0x7f4cd00cce10 2024-05-04T12:17:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.959+0000 7f4cceffd640 1 -- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 msgr2=0x7f4cd00b4660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T12:17:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.959+0000 7f4cceffd640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 0x7f4cd00b4660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.959+0000 7f4cceffd640 1 -- 172.21.15.52:0/1912267447 >> v2:172.21.15.111:3300/0 conn(0x7f4cd0160380 msgr2=0x7f4cd01607f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T12:17:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.959+0000 7f4cceffd640 1 --2- 172.21.15.52:0/1912267447 >> v2:172.21.15.111:3300/0 conn(0x7f4cd0160380 0x7f4cd01607f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.959+0000 7f4cceffd640 1 -- 172.21.15.52:0/1912267447 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd00cd920 con 0x7f4cd00cce10 2024-05-04T12:17:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.959+0000 7f4cceffd640 1 -- 172.21.15.52:0/1912267447 <== mon.1 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f4cc4001680 con 0x7f4cd00cce10 2024-05-04T12:17:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.959+0000 7f4cceffd640 1 -- 172.21.15.52:0/1912267447 >> v1:172.21.15.111:6789/0 conn(0x7f4cd00cce10 legacy=0x7f4cd015c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.963+0000 7f4cceffd640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cb80039c0 0x7f4cb8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.963+0000 7f4cceffd640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cb8004810 0x7f4cb8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.963+0000 7f4cceffd640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 0x7f4cb8005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.963+0000 7f4cceffd640 1 -- 172.21.15.52:0/1912267447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd00cdc00 con 0x7f4cb80039c0 2024-05-04T12:17:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.963+0000 7f4cceffd640 1 -- 172.21.15.52:0/1912267447 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd00cd640 con 0x7f4cb8004810 2024-05-04T12:17:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.963+0000 7f4cceffd640 1 -- 172.21.15.52:0/1912267447 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4cb800bbb0 con 0x7f4cd01531b0 2024-05-04T12:17:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.963+0000 7f4ccffff640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cb80039c0 0x7f4cb8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.963+0000 7f4ccf7fe640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cb8004810 0x7f4cb8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.963+0000 7f4cd4b4f640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 0x7f4cb8005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:57.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.963+0000 7f4ccffff640 1 -- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 msgr2=0x7f4cb8005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:17:57.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.963+0000 7f4ccffff640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd01531b0 0x7f4cb8005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:57.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.963+0000 7f4ccffff640 1 -- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cb8004810 msgr2=0x7f4cb8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:17:57.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.963+0000 7f4ccffff640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cb8004810 0x7f4cb8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:57.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.963+0000 7f4ccffff640 1 -- 172.21.15.52:0/1912267447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb8003530 con 0x7f4cb80039c0 2024-05-04T12:17:57.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.967+0000 7f4ccffff640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cb80039c0 0x7f4cb8003de0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000e900 tx=0x7f4cc001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5606343e0a8832c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:17:57.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.967+0000 7f4cceffd640 1 -- 172.21.15.52:0/1912267447 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc00ab070 con 0x7f4cb80039c0 2024-05-04T12:17:57.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.967+0000 7f4cceffd640 1 -- 172.21.15.52:0/1912267447 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc00a5720 con 0x7f4cb80039c0 2024-05-04T12:17:57.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.967+0000 7f4cceffd640 1 -- 172.21.15.52:0/1912267447 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc000f040 con 0x7f4cb80039c0 2024-05-04T12:17:57.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.967+0000 7f4cd65b2640 1 -- 172.21.15.52:0/1912267447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cb80039c0 msgr2=0x7f4cb8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:57.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.967+0000 7f4cd65b2640 1 --2- 172.21.15.52:0/1912267447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cb80039c0 0x7f4cb8003de0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000e900 tx=0x7f4cc001c930 comp rx=0 tx=0).stop 2024-05-04T12:17:57.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.967+0000 7f4cd4b4f640 1 -- 172.21.15.52:0/1912267447 reap_dead start 2024-05-04T12:17:57.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.967+0000 7f4cd65b2640 1 -- 172.21.15.52:0/1912267447 shutdown_connections 2024-05-04T12:17:57.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.967+0000 7f4cd65b2640 1 -- 172.21.15.52:0/1912267447 >> 172.21.15.52:0/1912267447 conn(0x7f4cd0070690 msgr2=0x7f4cd0070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:17:57.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.967+0000 7f4cd65b2640 1 -- 172.21.15.52:0/1912267447 shutdown_connections 2024-05-04T12:17:57.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.967+0000 7f4cd65b2640 1 -- 172.21.15.52:0/1912267447 wait complete. 2024-05-04T12:17:57.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.971+0000 7f4cd65b2640 1 Processor -- start 2024-05-04T12:17:57.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.971+0000 7f4cd65b2640 1 -- start start 2024-05-04T12:17:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd65b2640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd013d030 0x7f4cd014d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd65b2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cd01548e0 0x7f4cd0154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd65b2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cd00b0190 0x7f4cd00b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd65b2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd006d350 con 0x7f4cd00b0190 2024-05-04T12:17:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd65b2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd00b4c80 con 0x7f4cd01548e0 2024-05-04T12:17:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd65b2640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd006d4d0 con 0x7f4cd013d030 2024-05-04T12:17:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd4b4f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cd00b0190 0x7f4cd00b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd4b4f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cd00b0190 0x7f4cd00b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51764/0 (socket says 172.21.15.52:51764) 2024-05-04T12:17:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd4b4f640 1 -- 172.21.15.52:0/3444004035 learned_addr learned my addr 172.21.15.52:0/3444004035 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:17:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4ccf7fe640 1 --2- 172.21.15.52:0/3444004035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cd01548e0 0x7f4cd0154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:57.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4ccffff640 1 --2- 172.21.15.52:0/3444004035 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd013d030 0x7f4cd014d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:57.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd4b4f640 1 -- 172.21.15.52:0/3444004035 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd013d030 msgr2=0x7f4cd014d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:57.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd4b4f640 1 --2- 172.21.15.52:0/3444004035 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd013d030 0x7f4cd014d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:57.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd4b4f640 1 -- 172.21.15.52:0/3444004035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cd01548e0 msgr2=0x7f4cd0154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:57.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd4b4f640 1 --2- 172.21.15.52:0/3444004035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cd01548e0 0x7f4cd0154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:57.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd4b4f640 1 -- 172.21.15.52:0/3444004035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd01f3d20 con 0x7f4cd00b0190 2024-05-04T12:17:57.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd4b4f640 1 --2- 172.21.15.52:0/3444004035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cd00b0190 0x7f4cd00b0610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4cd015d8b0 tx=0x7f4cc8007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:17:57.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cccff9640 1 -- 172.21.15.52:0/3444004035 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc801a020 con 0x7f4cd00b0190 2024-05-04T12:17:57.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cccff9640 1 -- 172.21.15.52:0/3444004035 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc8016460 con 0x7f4cd00b0190 2024-05-04T12:17:57.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd65b2640 1 -- 172.21.15.52:0/3444004035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd01f4000 con 0x7f4cd00b0190 2024-05-04T12:17:57.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd65b2640 1 -- 172.21.15.52:0/3444004035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd01f45c0 con 0x7f4cd00b0190 2024-05-04T12:17:57.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cccff9640 1 -- 172.21.15.52:0/3444004035 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8016850 con 0x7f4cd00b0190 2024-05-04T12:17:57.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cd65b2640 1 -- 172.21.15.52:0/3444004035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb00052a0 con 0x7f4cd00b0190 2024-05-04T12:17:57.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cccff9640 1 -- 172.21.15.52:0/3444004035 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4cc801d020 con 0x7f4cd00b0190 2024-05-04T12:17:57.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.975+0000 7f4cccff9640 1 -- 172.21.15.52:0/3444004035 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4cc801fd10 con 0x7f4cd00b0190 2024-05-04T12:17:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:57.979+0000 7f4cccff9640 1 -- 172.21.15.52:0/3444004035 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4cc8016d80 con 0x7f4cd00b0190 2024-05-04T12:17:58.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:58.047+0000 7f4cd65b2640 1 -- 172.21.15.52:0/3444004035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f4cb0005860 con 0x7f4cd00b0190 2024-05-04T12:17:58.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:58.051+0000 7f4cccff9640 1 -- 172.21.15.52:0/3444004035 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f4cc801f410 con 0x7f4cd00b0190 2024-05-04T12:17:58.050 INFO:teuthology.orchestra.run.smithi052.stderr:adjusted tunables profile to default 2024-05-04T12:17:58.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:58.051+0000 7f4cd65b2640 1 -- 172.21.15.52:0/3444004035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cd00b0190 msgr2=0x7f4cd00b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:58.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:58.051+0000 7f4cd65b2640 1 --2- 172.21.15.52:0/3444004035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cd00b0190 0x7f4cd00b0610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4cd015d8b0 tx=0x7f4cc8007700 comp rx=0 tx=0).stop 2024-05-04T12:17:58.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:58.051+0000 7f4cd65b2640 1 -- 172.21.15.52:0/3444004035 shutdown_connections 2024-05-04T12:17:58.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:58.051+0000 7f4cd65b2640 1 --2- 172.21.15.52:0/3444004035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cd00b0190 0x7f4cd00b0610 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:58.051+0000 7f4cd65b2640 1 --2- 172.21.15.52:0/3444004035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cd01548e0 0x7f4cd0154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:58.051+0000 7f4cd65b2640 1 --2- 172.21.15.52:0/3444004035 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cd013d030 0x7f4cd014d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:58.051+0000 7f4cd65b2640 1 -- 172.21.15.52:0/3444004035 >> 172.21.15.52:0/3444004035 conn(0x7f4cd0070690 msgr2=0x7f4cd0072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:17:58.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:58.051+0000 7f4cd65b2640 1 -- 172.21.15.52:0/3444004035 shutdown_connections 2024-05-04T12:17:58.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:17:58.051+0000 7f4cd65b2640 1 -- 172.21.15.52:0/3444004035 wait complete. 2024-05-04T12:17:58.070 INFO:tasks.ceph:check_enable_crimson: False 2024-05-04T12:17:58.070 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-04T12:17:58.070 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:58.070 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-04T12:17:58.081 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:58.082 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-04T12:17:58.142 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:58.142 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-04T12:17:58.189 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:17:58.191+0000 7fcc39d6e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-04T12:17:58.202 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:17:58.202 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-04T12:17:58.258 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:17:58.259+0000 7f5767e52180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-04T12:17:58.261 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:58.261 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-04T12:17:58.274 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:58.274 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-04T12:17:58.329 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:58.329 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-04T12:17:58.382 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:17:58.382 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-04T12:17:58.438 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new b7012e43-90ab-42b8-b285-d5dc8e7f23e1 0 2024-05-04T12:17:58.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f71234d3640 1 Processor -- start 2024-05-04T12:17:58.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f71234d3640 1 -- start start 2024-05-04T12:17:58.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f71234d3640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f711c15ae10 0x7f711c0cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:58.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f71234d3640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f711c0ce3d0 0x7f711c06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:58.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f71234d3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f711c15b1f0 0x7f711c06bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:58.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f71234d3640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f711c0ce8e0 con 0x7f711c0ce3d0 2024-05-04T12:17:58.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f71234d3640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f711c06e270 con 0x7f711c15b1f0 2024-05-04T12:17:58.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f71234d3640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f711c06e550 con 0x7f711c15ae10 2024-05-04T12:17:58.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f712126f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f711c15ae10 0x7f711c0cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:58.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f7121a70640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f711c15b1f0 0x7f711c06bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:58.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f712126f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f711c15ae10 0x7f711c0cde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:38054/0 (socket says 172.21.15.111:38054) 2024-05-04T12:17:58.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f712126f640 1 -- 172.21.15.111:0/1445215446 learned_addr learned my addr 172.21.15.111:0/1445215446 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-04T12:17:58.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f7120a6e640 1 --2- 172.21.15.111:0/1445215446 >> v2:172.21.15.52:3300/0 conn(0x7f711c0ce3d0 0x7f711c06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:58.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f712126f640 1 -- 172.21.15.111:0/1445215446 >> v2:172.21.15.111:3300/0 conn(0x7f711c15b1f0 msgr2=0x7f711c06bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:17:58.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f712126f640 1 --2- 172.21.15.111:0/1445215446 >> v2:172.21.15.111:3300/0 conn(0x7f711c15b1f0 0x7f711c06bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f712126f640 1 -- 172.21.15.111:0/1445215446 >> v2:172.21.15.52:3300/0 conn(0x7f711c0ce3d0 msgr2=0x7f711c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:17:58.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f712126f640 1 --2- 172.21.15.111:0/1445215446 >> v2:172.21.15.52:3300/0 conn(0x7f711c0ce3d0 0x7f711c06b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f712126f640 1 -- 172.21.15.111:0/1445215446 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c06e830 con 0x7f711c15ae10 2024-05-04T12:17:58.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f7121a70640 1 --2- 172.21.15.111:0/1445215446 >> v2:172.21.15.111:3300/0 conn(0x7f711c15b1f0 0x7f711c06bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:17:58.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.603+0000 7f7120a6e640 1 --2- 172.21.15.111:0/1445215446 >> v2:172.21.15.52:3300/0 conn(0x7f711c0ce3d0 0x7f711c06b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:17:58.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f712126f640 1 --2- 172.21.15.111:0/1445215446 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f711c15ae10 0x7f711c0cde90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f711c0b1570 tx=0x7f711401c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6927662f8ee4e34 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:17:58.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f710ffff640 1 -- 172.21.15.111:0/1445215446 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71140a5800 con 0x7f711c15ae10 2024-05-04T12:17:58.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f710ffff640 1 -- 172.21.15.111:0/1445215446 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71140a59a0 con 0x7f711c15ae10 2024-05-04T12:17:58.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f710ffff640 1 -- 172.21.15.111:0/1445215446 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71140a5d70 con 0x7f711c15ae10 2024-05-04T12:17:58.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f71234d3640 1 -- 172.21.15.111:0/1445215446 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f711c15ae10 msgr2=0x7f711c0cde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:58.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f71234d3640 1 --2- 172.21.15.111:0/1445215446 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f711c15ae10 0x7f711c0cde90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f711c0b1570 tx=0x7f711401c960 comp rx=0 tx=0).stop 2024-05-04T12:17:58.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f71234d3640 1 -- 172.21.15.111:0/1445215446 shutdown_connections 2024-05-04T12:17:58.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f71234d3640 1 --2- 172.21.15.111:0/1445215446 >> v2:172.21.15.111:3300/0 conn(0x7f711c15b1f0 0x7f711c06bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f71234d3640 1 --2- 172.21.15.111:0/1445215446 >> v2:172.21.15.52:3300/0 conn(0x7f711c0ce3d0 0x7f711c06b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f71234d3640 1 --2- 172.21.15.111:0/1445215446 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f711c15ae10 0x7f711c0cde90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f71234d3640 1 -- 172.21.15.111:0/1445215446 >> 172.21.15.111:0/1445215446 conn(0x7f711c0d1460 msgr2=0x7f711c0d1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:17:58.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f71234d3640 1 -- 172.21.15.111:0/1445215446 shutdown_connections 2024-05-04T12:17:58.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f71234d3640 1 -- 172.21.15.111:0/1445215446 wait complete. 2024-05-04T12:17:58.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f71234d3640 1 Processor -- start 2024-05-04T12:17:58.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.607+0000 7f71234d3640 1 -- start start 2024-05-04T12:17:58.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f71234d3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f711c0ce3d0 0x7f711c1438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:58.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f71234d3640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f711c15ae10 0x7f711c133f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:58.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f71234d3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f711c15b1f0 0x7f711c1fa020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:58.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f71234d3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f711c15f550 con 0x7f711c0ce3d0 2024-05-04T12:17:58.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f71234d3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f711c15f3d0 con 0x7f711c15b1f0 2024-05-04T12:17:58.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f71234d3640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f711c15f6d0 con 0x7f711c15ae10 2024-05-04T12:17:58.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f7121a70640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f711c15b1f0 0x7f711c1fa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:58.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f7121a70640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f711c15b1f0 0x7f711c1fa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43076/0 (socket says 172.21.15.111:43076) 2024-05-04T12:17:58.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f7120a6e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f711c15ae10 0x7f711c133f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:58.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f7121a70640 1 -- 172.21.15.111:0/916613435 learned_addr learned my addr 172.21.15.111:0/916613435 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-04T12:17:58.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f712126f640 1 --2- 172.21.15.111:0/916613435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f711c0ce3d0 0x7f711c1438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:58.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f7121a70640 1 -- 172.21.15.111:0/916613435 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f711c15ae10 msgr2=0x7f711c133f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:58.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f7121a70640 1 --2- 172.21.15.111:0/916613435 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f711c15ae10 0x7f711c133f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f7121a70640 1 -- 172.21.15.111:0/916613435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f711c0ce3d0 msgr2=0x7f711c1438c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:58.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f7121a70640 1 --2- 172.21.15.111:0/916613435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f711c0ce3d0 0x7f711c1438c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f7121a70640 1 -- 172.21.15.111:0/916613435 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c143f80 con 0x7f711c15b1f0 2024-05-04T12:17:58.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f7121a70640 1 --2- 172.21.15.111:0/916613435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f711c15b1f0 0x7f711c1fa020 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f711800e930 tx=0x7f711800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:17:58.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f710dffb640 1 -- 172.21.15.111:0/916613435 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118017070 con 0x7f711c15b1f0 2024-05-04T12:17:58.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f710dffb640 1 -- 172.21.15.111:0/916613435 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7118004020 con 0x7f711c15b1f0 2024-05-04T12:17:58.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f71234d3640 1 -- 172.21.15.111:0/916613435 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711c144260 con 0x7f711c15b1f0 2024-05-04T12:17:58.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f710dffb640 1 -- 172.21.15.111:0/916613435 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118004430 con 0x7f711c15b1f0 2024-05-04T12:17:58.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f71234d3640 1 -- 172.21.15.111:0/916613435 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711c144820 con 0x7f711c15b1f0 2024-05-04T12:17:58.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f710dffb640 1 -- 172.21.15.111:0/916613435 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f711800c940 con 0x7f711c15b1f0 2024-05-04T12:17:58.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f710dffb640 1 -- 172.21.15.111:0/916613435 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f7118013470 con 0x7f711c15b1f0 2024-05-04T12:17:58.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.611+0000 7f71234d3640 1 -- 172.21.15.111:0/916613435 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f711c062e50 con 0x7f711c15b1f0 2024-05-04T12:17:58.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.615+0000 7f710dffb640 1 -- 172.21.15.111:0/916613435 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7118003da0 con 0x7f711c15b1f0 2024-05-04T12:17:58.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.683+0000 7f71234d3640 1 -- 172.21.15.111:0/916613435 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b7012e43-90ab-42b8-b285-d5dc8e7f23e1", "id": 0} v 0) -- 0x7f711c06c210 con 0x7f711c15b1f0 2024-05-04T12:17:58.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.691+0000 7f710dffb640 1 -- 172.21.15.111:0/916613435 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b7012e43-90ab-42b8-b285-d5dc8e7f23e1", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f7118003620 con 0x7f711c15b1f0 2024-05-04T12:17:58.690 INFO:teuthology.orchestra.run.smithi111.stdout:0 2024-05-04T12:17:58.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.691+0000 7f71234d3640 1 -- 172.21.15.111:0/916613435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f711c15b1f0 msgr2=0x7f711c1fa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:58.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.691+0000 7f71234d3640 1 --2- 172.21.15.111:0/916613435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f711c15b1f0 0x7f711c1fa020 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f711800e930 tx=0x7f711800edf0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.691+0000 7f71234d3640 1 -- 172.21.15.111:0/916613435 shutdown_connections 2024-05-04T12:17:58.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.691+0000 7f71234d3640 1 --2- 172.21.15.111:0/916613435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f711c15b1f0 0x7f711c1fa020 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.691+0000 7f71234d3640 1 --2- 172.21.15.111:0/916613435 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f711c15ae10 0x7f711c133f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.691+0000 7f71234d3640 1 --2- 172.21.15.111:0/916613435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f711c0ce3d0 0x7f711c1438c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.691+0000 7f71234d3640 1 -- 172.21.15.111:0/916613435 >> 172.21.15.111:0/916613435 conn(0x7f711c0d1460 msgr2=0x7f711c15c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:17:58.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.691+0000 7f71234d3640 1 -- 172.21.15.111:0/916613435 shutdown_connections 2024-05-04T12:17:58.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.691+0000 7f71234d3640 1 -- 172.21.15.111:0/916613435 wait complete. 2024-05-04T12:17:58.703 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:17:58.703+0000 7fcc39d6e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-04T12:17:58.707 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new 1d96acd5-48ed-45ac-90df-dd9f646bcd7c 1 2024-05-04T12:17:58.744 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:17:58.743+0000 7f5767e52180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-04T12:17:58.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.827+0000 7fbe181a4640 1 Processor -- start 2024-05-04T12:17:58.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.827+0000 7fbe181a4640 1 -- start start 2024-05-04T12:17:58.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe181a4640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbe1006ecf0 0x7fbe1015c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:58.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe181a4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbe1015ca10 0x7fbe100cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:58.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe181a4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe100b5b60 con 0x7fbe1015c0f0 2024-05-04T12:17:58.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe181a4640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fbe100b5e40 con 0x7fbe1015ca10 2024-05-04T12:17:58.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe181a4640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fbe100b6120 con 0x7fbe1006ecf0 2024-05-04T12:17:58.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe1573f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbe1006ecf0 0x7fbe1015c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:58.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe15f40640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbe1015c0f0 0x7fbe100b5310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:57980/0 (socket says 172.21.15.111:57980) 2024-05-04T12:17:58.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe1573f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbe1006ecf0 0x7fbe1015c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:38082/0 (socket says 172.21.15.111:38082) 2024-05-04T12:17:58.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe15f40640 1 -- 172.21.15.111:0/2394728746 learned_addr learned my addr 172.21.15.111:0/2394728746 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-04T12:17:58.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe16741640 1 --2- 172.21.15.111:0/2394728746 >> v2:172.21.15.52:3300/0 conn(0x7fbe1015ca10 0x7fbe100cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:58.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe1573f640 1 -- 172.21.15.111:0/2394728746 >> v2:172.21.15.52:3300/0 conn(0x7fbe1015ca10 msgr2=0x7fbe100cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:17:58.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe1573f640 1 --2- 172.21.15.111:0/2394728746 >> v2:172.21.15.52:3300/0 conn(0x7fbe1015ca10 0x7fbe100cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe1573f640 1 -- 172.21.15.111:0/2394728746 >> v1:172.21.15.111:6789/0 conn(0x7fbe1015c0f0 legacy=0x7fbe100b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:58.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe1573f640 1 -- 172.21.15.111:0/2394728746 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe100b6400 con 0x7fbe1006ecf0 2024-05-04T12:17:58.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe1573f640 1 --2- 172.21.15.111:0/2394728746 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbe1006ecf0 0x7fbe1015c4d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe000097c0 tx=0x7fbe0001c230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a95ac7cc688300e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:17:58.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe14f3e640 1 -- 172.21.15.111:0/2394728746 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe000b0070 con 0x7fbe1006ecf0 2024-05-04T12:17:58.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe14f3e640 1 -- 172.21.15.111:0/2394728746 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe000ab600 con 0x7fbe1006ecf0 2024-05-04T12:17:58.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe14f3e640 1 -- 172.21.15.111:0/2394728746 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe000ab9d0 con 0x7fbe1006ecf0 2024-05-04T12:17:58.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe181a4640 1 -- 172.21.15.111:0/2394728746 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbe1006ecf0 msgr2=0x7fbe1015c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:58.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe181a4640 1 --2- 172.21.15.111:0/2394728746 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbe1006ecf0 0x7fbe1015c4d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe000097c0 tx=0x7fbe0001c230 comp rx=0 tx=0).stop 2024-05-04T12:17:58.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe181a4640 1 -- 172.21.15.111:0/2394728746 shutdown_connections 2024-05-04T12:17:58.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe181a4640 1 --2- 172.21.15.111:0/2394728746 >> v2:172.21.15.52:3300/0 conn(0x7fbe1015ca10 0x7fbe100cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe181a4640 1 --2- 172.21.15.111:0/2394728746 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbe1006ecf0 0x7fbe1015c4d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe181a4640 1 -- 172.21.15.111:0/2394728746 >> 172.21.15.111:0/2394728746 conn(0x7fbe1006b210 msgr2=0x7fbe1006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:17:58.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe181a4640 1 -- 172.21.15.111:0/2394728746 shutdown_connections 2024-05-04T12:17:58.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.831+0000 7fbe181a4640 1 -- 172.21.15.111:0/2394728746 wait complete. 2024-05-04T12:17:58.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe181a4640 1 Processor -- start 2024-05-04T12:17:58.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe181a4640 1 -- start start 2024-05-04T12:17:58.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe181a4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe1006ecf0 0x7fbe10150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:58.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe181a4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe1015c0f0 0x7fbe100af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:58.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe181a4640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbe1015ca10 0x7fbe1014c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:58.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe181a4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe100b7780 con 0x7fbe1006ecf0 2024-05-04T12:17:58.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe181a4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbe100b7600 con 0x7fbe1015c0f0 2024-05-04T12:17:58.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe181a4640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fbe10168cb0 con 0x7fbe1015ca10 2024-05-04T12:17:58.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe1573f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe1015c0f0 0x7fbe100af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:58.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe16741640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbe1015ca10 0x7fbe1014c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe1573f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe1015c0f0 0x7fbe100af370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43090/0 (socket says 172.21.15.111:43090) 2024-05-04T12:17:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe1573f640 1 -- 172.21.15.111:0/386176073 learned_addr learned my addr 172.21.15.111:0/386176073 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-04T12:17:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe15f40640 1 --2- 172.21.15.111:0/386176073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe1006ecf0 0x7fbe10150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe1573f640 1 -- 172.21.15.111:0/386176073 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbe1015ca10 msgr2=0x7fbe1014c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe1573f640 1 --2- 172.21.15.111:0/386176073 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbe1015ca10 0x7fbe1014c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe1573f640 1 -- 172.21.15.111:0/386176073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe1006ecf0 msgr2=0x7fbe10150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe1573f640 1 --2- 172.21.15.111:0/386176073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe1006ecf0 0x7fbe10150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe1573f640 1 -- 172.21.15.111:0/386176073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe10150970 con 0x7fbe1015c0f0 2024-05-04T12:17:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe16741640 1 --2- 172.21.15.111:0/386176073 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbe1015ca10 0x7fbe1014c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:17:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe1573f640 1 --2- 172.21.15.111:0/386176073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe1015c0f0 0x7fbe100af370 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbe00019e60 tx=0x7fbe0001aa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:17:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe067fc640 1 -- 172.21.15.111:0/386176073 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe000b0050 con 0x7fbe1015c0f0 2024-05-04T12:17:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe067fc640 1 -- 172.21.15.111:0/386176073 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe000bc070 con 0x7fbe1015c0f0 2024-05-04T12:17:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe181a4640 1 -- 172.21.15.111:0/386176073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe10150c50 con 0x7fbe1015c0f0 2024-05-04T12:17:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe067fc640 1 -- 172.21.15.111:0/386176073 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe000aae40 con 0x7fbe1015c0f0 2024-05-04T12:17:58.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe181a4640 1 -- 172.21.15.111:0/386176073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe10151210 con 0x7fbe1015c0f0 2024-05-04T12:17:58.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe067fc640 1 -- 172.21.15.111:0/386176073 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbe000ab4c0 con 0x7fbe1015c0f0 2024-05-04T12:17:58.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe067fc640 1 -- 172.21.15.111:0/386176073 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fbe000b7800 con 0x7fbe1015c0f0 2024-05-04T12:17:58.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.835+0000 7fbe181a4640 1 -- 172.21.15.111:0/386176073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe10062e50 con 0x7fbe1015c0f0 2024-05-04T12:17:58.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.839+0000 7fbe067fc640 1 -- 172.21.15.111:0/386176073 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbe100cf8c0 con 0x7fbe1015c0f0 2024-05-04T12:17:58.904 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:17:58.903+0000 7fcc39d6e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-04T12:17:58.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.907+0000 7fbe181a4640 1 -- 172.21.15.111:0/386176073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1d96acd5-48ed-45ac-90df-dd9f646bcd7c", "id": 1} v 0) -- 0x7fbe10075320 con 0x7fbe1015c0f0 2024-05-04T12:17:58.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.915+0000 7fbe067fc640 1 -- 172.21.15.111:0/386176073 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1d96acd5-48ed-45ac-90df-dd9f646bcd7c", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fbe000b7a00 con 0x7fbe1015c0f0 2024-05-04T12:17:58.914 INFO:teuthology.orchestra.run.smithi111.stdout:1 2024-05-04T12:17:58.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.915+0000 7fbe181a4640 1 -- 172.21.15.111:0/386176073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe1015c0f0 msgr2=0x7fbe100af370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:58.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.915+0000 7fbe181a4640 1 --2- 172.21.15.111:0/386176073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe1015c0f0 0x7fbe100af370 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbe00019e60 tx=0x7fbe0001aa70 comp rx=0 tx=0).stop 2024-05-04T12:17:58.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.915+0000 7fbe181a4640 1 -- 172.21.15.111:0/386176073 shutdown_connections 2024-05-04T12:17:58.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.915+0000 7fbe181a4640 1 --2- 172.21.15.111:0/386176073 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbe1015ca10 0x7fbe1014c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.915+0000 7fbe181a4640 1 --2- 172.21.15.111:0/386176073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe1015c0f0 0x7fbe100af370 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.915+0000 7fbe181a4640 1 --2- 172.21.15.111:0/386176073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe1006ecf0 0x7fbe10150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:58.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.915+0000 7fbe181a4640 1 -- 172.21.15.111:0/386176073 >> 172.21.15.111:0/386176073 conn(0x7fbe1006b210 msgr2=0x7fbe1006f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:17:58.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.915+0000 7fbe181a4640 1 -- 172.21.15.111:0/386176073 shutdown_connections 2024-05-04T12:17:58.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:58.915+0000 7fbe181a4640 1 -- 172.21.15.111:0/386176073 wait complete. 2024-05-04T12:17:58.932 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new 6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517 2 2024-05-04T12:17:59.016 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:17:59.015+0000 7f5767e52180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-04T12:17:59.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.099+0000 7fb380fa8640 1 Processor -- start 2024-05-04T12:17:59.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.099+0000 7fb380fa8640 1 -- start start 2024-05-04T12:17:59.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb380fa8640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb37c15a930 0x7fb37c0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb380fa8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb37c0aecd0 0x7fb37c072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb380fa8640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb37c0d0080 con 0x7fb37c0af220 2024-05-04T12:17:59.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb380fa8640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fb37c073470 con 0x7fb37c0aecd0 2024-05-04T12:17:59.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb380fa8640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fb37c06f320 con 0x7fb37c15a930 2024-05-04T12:17:59.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb379d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb37c0aecd0 0x7fb37c072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb37a59c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb37c15a930 0x7fb37c0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb37ad9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb37c0af220 0x7fb37c06da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:57984/0 (socket says 172.21.15.111:57984) 2024-05-04T12:17:59.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb379d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb37c0aecd0 0x7fb37c072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43106/0 (socket says 172.21.15.111:43106) 2024-05-04T12:17:59.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb37a59c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb37c15a930 0x7fb37c0729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:38096/0 (socket says 172.21.15.111:38096) 2024-05-04T12:17:59.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb37ad9d640 1 -- 172.21.15.111:0/3487134657 learned_addr learned my addr 172.21.15.111:0/3487134657 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-04T12:17:59.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb37a59c640 1 -- 172.21.15.111:0/3487134657 >> v2:172.21.15.111:3300/0 conn(0x7fb37c0aecd0 msgr2=0x7fb37c072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:17:59.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb37a59c640 1 --2- 172.21.15.111:0/3487134657 >> v2:172.21.15.111:3300/0 conn(0x7fb37c0aecd0 0x7fb37c072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb37a59c640 1 -- 172.21.15.111:0/3487134657 >> v1:172.21.15.111:6789/0 conn(0x7fb37c0af220 legacy=0x7fb37c06da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb37a59c640 1 -- 172.21.15.111:0/3487134657 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c06f600 con 0x7fb37c15a930 2024-05-04T12:17:59.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb379d9b640 1 --2- 172.21.15.111:0/3487134657 >> v2:172.21.15.111:3300/0 conn(0x7fb37c0aecd0 0x7fb37c072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:17:59.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb37a59c640 1 --2- 172.21.15.111:0/3487134657 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb37c15a930 0x7fb37c0729f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00aea0 tx=0x7fb36c01d9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b891bfe9ef590227 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:17:59.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb37959a640 1 -- 172.21.15.111:0/3487134657 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0a67f0 con 0x7fb37c15a930 2024-05-04T12:17:59.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb37959a640 1 -- 172.21.15.111:0/3487134657 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c0a6990 con 0x7fb37c15a930 2024-05-04T12:17:59.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.103+0000 7fb37959a640 1 -- 172.21.15.111:0/3487134657 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0a6d60 con 0x7fb37c15a930 2024-05-04T12:17:59.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 -- 172.21.15.111:0/3487134657 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb37c15a930 msgr2=0x7fb37c0729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 --2- 172.21.15.111:0/3487134657 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb37c15a930 0x7fb37c0729f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00aea0 tx=0x7fb36c01d9b0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 -- 172.21.15.111:0/3487134657 shutdown_connections 2024-05-04T12:17:59.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 --2- 172.21.15.111:0/3487134657 >> v2:172.21.15.111:3300/0 conn(0x7fb37c0aecd0 0x7fb37c072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 --2- 172.21.15.111:0/3487134657 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb37c15a930 0x7fb37c0729f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 -- 172.21.15.111:0/3487134657 >> 172.21.15.111:0/3487134657 conn(0x7fb37c0b2ed0 msgr2=0x7fb37c0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:17:59.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 -- 172.21.15.111:0/3487134657 shutdown_connections 2024-05-04T12:17:59.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 -- 172.21.15.111:0/3487134657 wait complete. 2024-05-04T12:17:59.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 Processor -- start 2024-05-04T12:17:59.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 -- start start 2024-05-04T12:17:59.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb37c0aecd0 0x7fb37c15c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb37c0af220 0x7fb37c147980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb37c15a930 0x7fb37c151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c15ad10 con 0x7fb37c0af220 2024-05-04T12:17:59.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c06efb0 con 0x7fb37c15a930 2024-05-04T12:17:59.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fb37c15ae90 con 0x7fb37c0aecd0 2024-05-04T12:17:59.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb37a59c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb37c0aecd0 0x7fb37c15c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb37a59c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb37c0aecd0 0x7fb37c15c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:38110/0 (socket says 172.21.15.111:38110) 2024-05-04T12:17:59.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb37a59c640 1 -- 172.21.15.111:0/1612146961 learned_addr learned my addr 172.21.15.111:0/1612146961 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-04T12:17:59.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb37ad9d640 1 --2- 172.21.15.111:0/1612146961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb37c15a930 0x7fb37c151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb37a59c640 1 -- 172.21.15.111:0/1612146961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb37c15a930 msgr2=0x7fb37c151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb37a59c640 1 --2- 172.21.15.111:0/1612146961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb37c15a930 0x7fb37c151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb37a59c640 1 -- 172.21.15.111:0/1612146961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb37c0af220 msgr2=0x7fb37c147980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb379d9b640 1 --2- 172.21.15.111:0/1612146961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb37c0af220 0x7fb37c147980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb37a59c640 1 --2- 172.21.15.111:0/1612146961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb37c0af220 0x7fb37c147980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb37a59c640 1 -- 172.21.15.111:0/1612146961 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c15cd00 con 0x7fb37c0aecd0 2024-05-04T12:17:59.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb37a59c640 1 --2- 172.21.15.111:0/1612146961 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb37c0aecd0 0x7fb37c15c700 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0b0040 tx=0x7fb36c01a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:17:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb36affd640 1 -- 172.21.15.111:0/1612146961 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0bd030 con 0x7fb37c0aecd0 2024-05-04T12:17:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb36affd640 1 -- 172.21.15.111:0/1612146961 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c0b9470 con 0x7fb37c0aecd0 2024-05-04T12:17:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 -- 172.21.15.111:0/1612146961 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37c15cfe0 con 0x7fb37c0aecd0 2024-05-04T12:17:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb36affd640 1 -- 172.21.15.111:0/1612146961 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0b9840 con 0x7fb37c0aecd0 2024-05-04T12:17:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb380fa8640 1 -- 172.21.15.111:0/1612146961 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37c15d5a0 con 0x7fb37c0aecd0 2024-05-04T12:17:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb36affd640 1 -- 172.21.15.111:0/1612146961 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb36c0c0070 con 0x7fb37c0aecd0 2024-05-04T12:17:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.107+0000 7fb36affd640 1 -- 172.21.15.111:0/1612146961 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fb36c0c2ab0 con 0x7fb37c0aecd0 2024-05-04T12:17:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.111+0000 7fb380fa8640 1 -- 172.21.15.111:0/1612146961 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb37c062e50 con 0x7fb37c0aecd0 2024-05-04T12:17:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.115+0000 7fb36affd640 1 -- 172.21.15.111:0/1612146961 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb36c0c7070 con 0x7fb37c0aecd0 2024-05-04T12:17:59.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.183+0000 7fb380fa8640 1 -- 172.21.15.111:0/1612146961 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517", "id": 2} v 0) -- 0x7fb37c075320 con 0x7fb37c0aecd0 2024-05-04T12:17:59.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.191+0000 7fb36affd640 1 -- 172.21.15.111:0/1612146961 <== mon.2 v2:172.21.15.111:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fb36c017660 con 0x7fb37c0aecd0 2024-05-04T12:17:59.191 INFO:teuthology.orchestra.run.smithi111.stdout:2 2024-05-04T12:17:59.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.191+0000 7fb380fa8640 1 -- 172.21.15.111:0/1612146961 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb37c0aecd0 msgr2=0x7fb37c15c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.191+0000 7fb380fa8640 1 --2- 172.21.15.111:0/1612146961 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb37c0aecd0 0x7fb37c15c700 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0b0040 tx=0x7fb36c01a750 comp rx=0 tx=0).stop 2024-05-04T12:17:59.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.191+0000 7fb380fa8640 1 -- 172.21.15.111:0/1612146961 shutdown_connections 2024-05-04T12:17:59.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.191+0000 7fb380fa8640 1 --2- 172.21.15.111:0/1612146961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb37c15a930 0x7fb37c151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.191+0000 7fb380fa8640 1 --2- 172.21.15.111:0/1612146961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb37c0af220 0x7fb37c147980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.191+0000 7fb380fa8640 1 --2- 172.21.15.111:0/1612146961 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb37c0aecd0 0x7fb37c15c700 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.191+0000 7fb380fa8640 1 -- 172.21.15.111:0/1612146961 >> 172.21.15.111:0/1612146961 conn(0x7fb37c0b2ed0 msgr2=0x7fb37c0b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:17:59.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.191+0000 7fb380fa8640 1 -- 172.21.15.111:0/1612146961 shutdown_connections 2024-05-04T12:17:59.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.191+0000 7fb380fa8640 1 -- 172.21.15.111:0/1612146961 wait complete. 2024-05-04T12:17:59.208 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new bf071341-8de8-48e7-9454-c7301ca20351 3 2024-05-04T12:17:59.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.371+0000 7fe976111640 1 Processor -- start 2024-05-04T12:17:59.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.371+0000 7fe976111640 1 -- start start 2024-05-04T12:17:59.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.371+0000 7fe976111640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe9700cca90 0x7fe970166d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.371+0000 7fe976111640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9700cece0 con 0x7fe9700ce620 2024-05-04T12:17:59.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.371+0000 7fe976111640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9700cefc0 con 0x7fe97014c1b0 2024-05-04T12:17:59.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.371+0000 7fe976111640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fe9700cf2a0 con 0x7fe9700cca90 2024-05-04T12:17:59.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.371+0000 7fe96f7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe97014c1b0 0x7fe970163680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:57998/0 (socket says 172.21.15.111:57998) 2024-05-04T12:17:59.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.371+0000 7fe96f7fe640 1 -- 172.21.15.111:0/371807069 learned_addr learned my addr 172.21.15.111:0/371807069 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-04T12:17:59.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 -- 172.21.15.111:0/371807069 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525188749 0 0) 0x7fe9700cefc0 con 0x7fe97014c1b0 2024-05-04T12:17:59.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96ffff640 1 --2- 172.21.15.111:0/371807069 >> v2:172.21.15.52:3300/0 conn(0x7fe9700cca90 0x7fe970166d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 -- 172.21.15.111:0/371807069 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe950003530 con 0x7fe97014c1b0 2024-05-04T12:17:59.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 -- 172.21.15.111:0/371807069 <== mon.1 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1941257058 0 0) 0x7fe950003530 con 0x7fe97014c1b0 2024-05-04T12:17:59.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 -- 172.21.15.111:0/371807069 >> v2:172.21.15.52:3300/0 conn(0x7fe9700cca90 msgr2=0x7fe970166d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:17:59.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 --2- 172.21.15.111:0/371807069 >> v2:172.21.15.52:3300/0 conn(0x7fe9700cca90 0x7fe970166d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 -- 172.21.15.111:0/371807069 >> v1:172.21.15.52:6789/0 conn(0x7fe9700ce620 legacy=0x7fe9700cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 -- 172.21.15.111:0/371807069 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe970162e80 con 0x7fe97014c1b0 2024-05-04T12:17:59.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 -- 172.21.15.111:0/371807069 <== mon.1 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7fe968001a00 con 0x7fe97014c1b0 2024-05-04T12:17:59.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 -- 172.21.15.111:0/371807069 >> v1:172.21.15.111:6789/0 conn(0x7fe97014c1b0 legacy=0x7fe970163680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 --2- 172.21.15.111:0/371807069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9500039c0 0x7fe950003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 --2- 172.21.15.111:0/371807069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe950004810 0x7fe950004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 --2- 172.21.15.111:0/371807069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9500051d0 0x7fe95000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 -- 172.21.15.111:0/371807069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe9700cf2a0 con 0x7fe9500039c0 2024-05-04T12:17:59.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 -- 172.21.15.111:0/371807069 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe95000c070 con 0x7fe950004810 2024-05-04T12:17:59.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 -- 172.21.15.111:0/371807069 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe95000c270 con 0x7fe9500051d0 2024-05-04T12:17:59.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96ffff640 1 --2- 172.21.15.111:0/371807069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9500051d0 0x7fe95000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96effd640 1 --2- 172.21.15.111:0/371807069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe950004810 0x7fe950004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96f7fe640 1 --2- 172.21.15.111:0/371807069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9500039c0 0x7fe950003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96ffff640 1 -- 172.21.15.111:0/371807069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe950004810 msgr2=0x7fe950004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:17:59.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96ffff640 1 --2- 172.21.15.111:0/371807069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe950004810 0x7fe950004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96ffff640 1 -- 172.21.15.111:0/371807069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9500039c0 msgr2=0x7fe950003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:17:59.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96ffff640 1 --2- 172.21.15.111:0/371807069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9500039c0 0x7fe950003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.378 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96ffff640 1 -- 172.21.15.111:0/371807069 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe950003530 con 0x7fe9500051d0 2024-05-04T12:17:59.378 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96f7fe640 1 --2- 172.21.15.111:0/371807069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9500039c0 0x7fe950003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:17:59.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96ffff640 1 -- 172.21.15.111:0/371807069 reap_dead start 2024-05-04T12:17:59.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96ffff640 1 --2- 172.21.15.111:0/371807069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9500051d0 0x7fe95000b9d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe97006f800 tx=0x7fe96001ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1037b62632506b45 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:17:59.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 -- 172.21.15.111:0/371807069 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9600096b0 con 0x7fe9500051d0 2024-05-04T12:17:59.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 -- 172.21.15.111:0/371807069 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9600a6430 con 0x7fe9500051d0 2024-05-04T12:17:59.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.375+0000 7fe96e7fc640 1 -- 172.21.15.111:0/371807069 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9600a6800 con 0x7fe9500051d0 2024-05-04T12:17:59.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 -- 172.21.15.111:0/371807069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9500051d0 msgr2=0x7fe95000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 --2- 172.21.15.111:0/371807069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9500051d0 0x7fe95000b9d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe97006f800 tx=0x7fe96001ca50 comp rx=0 tx=0).stop 2024-05-04T12:17:59.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 -- 172.21.15.111:0/371807069 shutdown_connections 2024-05-04T12:17:59.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 --2- 172.21.15.111:0/371807069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9500051d0 0x7fe95000b9d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 -- 172.21.15.111:0/371807069 >> 172.21.15.111:0/371807069 conn(0x7fe970070690 msgr2=0x7fe970070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:17:59.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 -- 172.21.15.111:0/371807069 shutdown_connections 2024-05-04T12:17:59.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 -- 172.21.15.111:0/371807069 wait complete. 2024-05-04T12:17:59.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 Processor -- start 2024-05-04T12:17:59.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 -- start start 2024-05-04T12:17:59.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9500051d0 0x7fe97013ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe970165fb0 0x7fe97014b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9701405d0 0x7fe970142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe970072540 con 0x7fe9701405d0 2024-05-04T12:17:59.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe9700723c0 con 0x7fe970165fb0 2024-05-04T12:17:59.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe9700726c0 con 0x7fe9500051d0 2024-05-04T12:17:59.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe96f7fe640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9500051d0 0x7fe97013ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe96f7fe640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9500051d0 0x7fe97013ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:38130/0 (socket says 172.21.15.111:38130) 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe96f7fe640 1 -- 172.21.15.111:0/2591026317 learned_addr learned my addr 172.21.15.111:0/2591026317 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe96effd640 1 --2- 172.21.15.111:0/2591026317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe970165fb0 0x7fe97014b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe96ffff640 1 --2- 172.21.15.111:0/2591026317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9701405d0 0x7fe970142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe96f7fe640 1 -- 172.21.15.111:0/2591026317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe970165fb0 msgr2=0x7fe97014b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe96f7fe640 1 --2- 172.21.15.111:0/2591026317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe970165fb0 0x7fe97014b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe96f7fe640 1 -- 172.21.15.111:0/2591026317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9701405d0 msgr2=0x7fe970142660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe96f7fe640 1 --2- 172.21.15.111:0/2591026317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9701405d0 0x7fe970142660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe96f7fe640 1 -- 172.21.15.111:0/2591026317 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97013f530 con 0x7fe9500051d0 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe96effd640 1 --2- 172.21.15.111:0/2591026317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe970165fb0 0x7fe97014b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe96f7fe640 1 --2- 172.21.15.111:0/2591026317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9500051d0 0x7fe97013ef50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe968002190 tx=0x7fe96800b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe967fff640 1 -- 172.21.15.111:0/2591026317 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968019070 con 0x7fe9500051d0 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe967fff640 1 -- 172.21.15.111:0/2591026317 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe968004d70 con 0x7fe9500051d0 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 -- 172.21.15.111:0/2591026317 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97013f810 con 0x7fe9500051d0 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe967fff640 1 -- 172.21.15.111:0/2591026317 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9680072f0 con 0x7fe9500051d0 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe976111640 1 -- 172.21.15.111:0/2591026317 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97013fdd0 con 0x7fe9500051d0 2024-05-04T12:17:59.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe967fff640 1 -- 172.21.15.111:0/2591026317 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe96801c070 con 0x7fe9500051d0 2024-05-04T12:17:59.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.379+0000 7fe967fff640 1 -- 172.21.15.111:0/2591026317 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fe96801d3b0 con 0x7fe9500051d0 2024-05-04T12:17:59.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.383+0000 7fe976111640 1 -- 172.21.15.111:0/2591026317 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe970062e50 con 0x7fe9500051d0 2024-05-04T12:17:59.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.383+0000 7fe967fff640 1 -- 172.21.15.111:0/2591026317 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe968023070 con 0x7fe9500051d0 2024-05-04T12:17:59.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.451+0000 7fe976111640 1 -- 172.21.15.111:0/2591026317 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bf071341-8de8-48e7-9454-c7301ca20351", "id": 3} v 0) -- 0x7fe970075320 con 0x7fe9500051d0 2024-05-04T12:17:59.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.459+0000 7fe967fff640 1 -- 172.21.15.111:0/2591026317 <== mon.2 v2:172.21.15.111:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bf071341-8de8-48e7-9454-c7301ca20351", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fe968013a10 con 0x7fe9500051d0 2024-05-04T12:17:59.458 INFO:teuthology.orchestra.run.smithi111.stdout:3 2024-05-04T12:17:59.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.459+0000 7fe976111640 1 -- 172.21.15.111:0/2591026317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9500051d0 msgr2=0x7fe97013ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.459+0000 7fe976111640 1 --2- 172.21.15.111:0/2591026317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9500051d0 0x7fe97013ef50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe968002190 tx=0x7fe96800b560 comp rx=0 tx=0).stop 2024-05-04T12:17:59.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.459+0000 7fe976111640 1 -- 172.21.15.111:0/2591026317 shutdown_connections 2024-05-04T12:17:59.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.459+0000 7fe976111640 1 --2- 172.21.15.111:0/2591026317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9701405d0 0x7fe970142660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.459+0000 7fe976111640 1 --2- 172.21.15.111:0/2591026317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe970165fb0 0x7fe97014b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.459+0000 7fe976111640 1 --2- 172.21.15.111:0/2591026317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9500051d0 0x7fe97013ef50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.459+0000 7fe976111640 1 -- 172.21.15.111:0/2591026317 >> 172.21.15.111:0/2591026317 conn(0x7fe970070690 msgr2=0x7fe970162480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:17:59.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.459+0000 7fe976111640 1 -- 172.21.15.111:0/2591026317 shutdown_connections 2024-05-04T12:17:59.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.459+0000 7fe976111640 1 -- 172.21.15.111:0/2591026317 wait complete. 2024-05-04T12:17:59.474 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new aa6cd584-fce4-4997-867d-587d65f3b0e0 4 2024-05-04T12:17:59.477 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:17:59.475+0000 7fcc39d6e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-04T12:17:59.533 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:17:59.531+0000 7fcc39d6e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-04T12:17:59.588 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:17:59.587+0000 7fcc39d6e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-04T12:17:59.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.591+0000 7f9408ae3640 1 Processor -- start 2024-05-04T12:17:59.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.591+0000 7f9408ae3640 1 -- start start 2024-05-04T12:17:59.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9408ae3640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9404073090 0x7f94040ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9408ae3640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f940406eb30 0x7f940406ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9408ae3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94040cd5a0 con 0x7f9404144130 2024-05-04T12:17:59.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9408ae3640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f940406d650 con 0x7f9404073090 2024-05-04T12:17:59.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9408ae3640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f94040cd880 con 0x7f940406eb30 2024-05-04T12:17:59.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f940259c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f9404144130 0x7f940415adf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58000/0 (socket says 172.21.15.111:58000) 2024-05-04T12:17:59.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f940259c640 1 -- 172.21.15.111:0/4233953627 learned_addr learned my addr 172.21.15.111:0/4233953627 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-04T12:17:59.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9402d9d640 1 --2- 172.21.15.111:0/4233953627 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f940406eb30 0x7f940406ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f93f9d9b640 1 --2- 172.21.15.111:0/4233953627 >> v2:172.21.15.52:3300/0 conn(0x7f9404073090 0x7f94040ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9401d9b640 1 -- 172.21.15.111:0/4233953627 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1537330345 0 0) 0x7f94040cd5a0 con 0x7f9404144130 2024-05-04T12:17:59.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9401d9b640 1 -- 172.21.15.111:0/4233953627 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93f0003530 con 0x7f9404144130 2024-05-04T12:17:59.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9402d9d640 1 -- 172.21.15.111:0/4233953627 >> v2:172.21.15.52:3300/0 conn(0x7f9404073090 msgr2=0x7f94040ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:17:59.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9402d9d640 1 --2- 172.21.15.111:0/4233953627 >> v2:172.21.15.52:3300/0 conn(0x7f9404073090 0x7f94040ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9402d9d640 1 -- 172.21.15.111:0/4233953627 >> v1:172.21.15.111:6789/0 conn(0x7f9404144130 legacy=0x7f940415adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9402d9d640 1 -- 172.21.15.111:0/4233953627 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94040cdb60 con 0x7f940406eb30 2024-05-04T12:17:59.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f93f9d9b640 1 --2- 172.21.15.111:0/4233953627 >> v2:172.21.15.52:3300/0 conn(0x7f9404073090 0x7f94040ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:17:59.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9402d9d640 1 --2- 172.21.15.111:0/4233953627 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f940406eb30 0x7f940406ef40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0097c0 tx=0x7f93fc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ce72d482700965a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:17:59.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9401d9b640 1 -- 172.21.15.111:0/4233953627 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc0ab620 con 0x7f940406eb30 2024-05-04T12:17:59.598 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9401d9b640 1 -- 172.21.15.111:0/4233953627 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93fc0b0070 con 0x7f940406eb30 2024-05-04T12:17:59.598 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.595+0000 7f9401d9b640 1 -- 172.21.15.111:0/4233953627 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc0ab9e0 con 0x7f940406eb30 2024-05-04T12:17:59.598 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 -- 172.21.15.111:0/4233953627 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f940406eb30 msgr2=0x7f940406ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 --2- 172.21.15.111:0/4233953627 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f940406eb30 0x7f940406ef40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0097c0 tx=0x7f93fc01c4a0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 -- 172.21.15.111:0/4233953627 shutdown_connections 2024-05-04T12:17:59.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 --2- 172.21.15.111:0/4233953627 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f940406eb30 0x7f940406ef40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 --2- 172.21.15.111:0/4233953627 >> v2:172.21.15.52:3300/0 conn(0x7f9404073090 0x7f94040ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 -- 172.21.15.111:0/4233953627 >> 172.21.15.111:0/4233953627 conn(0x7f9404070690 msgr2=0x7f9404070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:17:59.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 -- 172.21.15.111:0/4233953627 shutdown_connections 2024-05-04T12:17:59.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 -- 172.21.15.111:0/4233953627 wait complete. 2024-05-04T12:17:59.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 Processor -- start 2024-05-04T12:17:59.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 -- start start 2024-05-04T12:17:59.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f940406eb30 0x7f9404139890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404073090 0x7f940413ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9404144130 0x7f9404157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f94040cee20 con 0x7f9404073090 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f940406f480 con 0x7f9404144130 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f94040cefa0 con 0x7f940406eb30 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9402d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9404144130 0x7f9404157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f940259c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f940406eb30 0x7f9404139890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9402d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9404144130 0x7f9404157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43134/0 (socket says 172.21.15.111:43134) 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9402d9d640 1 -- 172.21.15.111:0/903877302 learned_addr learned my addr 172.21.15.111:0/903877302 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f93f9d9b640 1 --2- 172.21.15.111:0/903877302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404073090 0x7f940413ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9402d9d640 1 -- 172.21.15.111:0/903877302 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f940406eb30 msgr2=0x7f9404139890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9402d9d640 1 --2- 172.21.15.111:0/903877302 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f940406eb30 0x7f9404139890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9402d9d640 1 -- 172.21.15.111:0/903877302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404073090 msgr2=0x7f940413ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9402d9d640 1 --2- 172.21.15.111:0/903877302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404073090 0x7f940413ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9402d9d640 1 -- 172.21.15.111:0/903877302 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94041f6c60 con 0x7f9404144130 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f93f9d9b640 1 --2- 172.21.15.111:0/903877302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404073090 0x7f940413ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9402d9d640 1 --2- 172.21.15.111:0/903877302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9404144130 0x7f9404157f00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f93fc01a0d0 tx=0x7f93fc01ace0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:17:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f93fb7fe640 1 -- 172.21.15.111:0/903877302 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc0abb30 con 0x7f9404144130 2024-05-04T12:17:59.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f93fb7fe640 1 -- 172.21.15.111:0/903877302 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93fc0abcd0 con 0x7f9404144130 2024-05-04T12:17:59.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 -- 172.21.15.111:0/903877302 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9404140b70 con 0x7f9404144130 2024-05-04T12:17:59.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f93fb7fe640 1 -- 172.21.15.111:0/903877302 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc0bf630 con 0x7f9404144130 2024-05-04T12:17:59.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f9408ae3640 1 -- 172.21.15.111:0/903877302 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9404141130 con 0x7f9404144130 2024-05-04T12:17:59.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f93fb7fe640 1 -- 172.21.15.111:0/903877302 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f93fc0bd020 con 0x7f9404144130 2024-05-04T12:17:59.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.599+0000 7f93fb7fe640 1 -- 172.21.15.111:0/903877302 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f93fc0c83f0 con 0x7f9404144130 2024-05-04T12:17:59.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.603+0000 7f9408ae3640 1 -- 172.21.15.111:0/903877302 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9404062e50 con 0x7f9404144130 2024-05-04T12:17:59.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.603+0000 7f93fb7fe640 1 -- 172.21.15.111:0/903877302 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f94041f6c60 con 0x7f9404144130 2024-05-04T12:17:59.617 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:17:59.615+0000 7f5767e52180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-04T12:17:59.668 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:17:59.667+0000 7fcc39d6e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-04T12:17:59.672 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:17:59.671+0000 7f5767e52180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-04T12:17:59.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.671+0000 7f9408ae3640 1 -- 172.21.15.111:0/903877302 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd new", "uuid": "aa6cd584-fce4-4997-867d-587d65f3b0e0", "id": 4} v 0) -- 0x7f9404075320 con 0x7f9404144130 2024-05-04T12:17:59.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.675+0000 7f93fb7fe640 1 -- 172.21.15.111:0/903877302 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aa6cd584-fce4-4997-867d-587d65f3b0e0", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f93fc016090 con 0x7f9404144130 2024-05-04T12:17:59.678 INFO:teuthology.orchestra.run.smithi111.stdout:4 2024-05-04T12:17:59.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.679+0000 7f9408ae3640 1 -- 172.21.15.111:0/903877302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9404144130 msgr2=0x7f9404157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.679+0000 7f9408ae3640 1 --2- 172.21.15.111:0/903877302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9404144130 0x7f9404157f00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f93fc01a0d0 tx=0x7f93fc01ace0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.679+0000 7f9408ae3640 1 -- 172.21.15.111:0/903877302 shutdown_connections 2024-05-04T12:17:59.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.679+0000 7f9408ae3640 1 --2- 172.21.15.111:0/903877302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9404144130 0x7f9404157f00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.679+0000 7f9408ae3640 1 --2- 172.21.15.111:0/903877302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404073090 0x7f940413ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.679+0000 7f9408ae3640 1 --2- 172.21.15.111:0/903877302 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f940406eb30 0x7f9404139890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.679+0000 7f9408ae3640 1 -- 172.21.15.111:0/903877302 >> 172.21.15.111:0/903877302 conn(0x7f9404070690 msgr2=0x7f9404159fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:17:59.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.679+0000 7f9408ae3640 1 -- 172.21.15.111:0/903877302 shutdown_connections 2024-05-04T12:17:59.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.679+0000 7f9408ae3640 1 -- 172.21.15.111:0/903877302 wait complete. 2024-05-04T12:17:59.694 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new 7115a071-f397-4df5-b177-83637c288ff1 5 2024-05-04T12:17:59.723 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:17:59.723+0000 7fcc39d6e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-04T12:17:59.727 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:17:59.727+0000 7f5767e52180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-04T12:17:59.803 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:17:59.803+0000 7fcc39d6e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-04T12:17:59.806 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:17:59.807+0000 7f5767e52180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-04T12:17:59.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.807+0000 7f0c252c0640 1 Processor -- start 2024-05-04T12:17:59.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.807+0000 7f0c252c0640 1 -- start start 2024-05-04T12:17:59.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c252c0640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c2015e080 0x7f0c200b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c252c0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0c20162560 0x7f0c20072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c252c0640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c200cea90 con 0x7f0c2015e460 2024-05-04T12:17:59.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c252c0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f0c20071ed0 con 0x7f0c20162560 2024-05-04T12:17:59.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c252c0640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f0c200b6300 con 0x7f0c2015e080 2024-05-04T12:17:59.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c1effd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c2015e080 0x7f0c200b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c1effd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c2015e080 0x7f0c200b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:38154/0 (socket says 172.21.15.111:38154) 2024-05-04T12:17:59.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c1effd640 1 -- 172.21.15.111:0/2095161123 learned_addr learned my addr 172.21.15.111:0/2095161123 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-04T12:17:59.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c1f7fe640 1 --2- 172.21.15.111:0/2095161123 >> v2:172.21.15.111:3300/0 conn(0x7f0c20162560 0x7f0c20072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c1effd640 1 -- 172.21.15.111:0/2095161123 >> v2:172.21.15.111:3300/0 conn(0x7f0c20162560 msgr2=0x7f0c20072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:17:59.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c1effd640 1 --2- 172.21.15.111:0/2095161123 >> v2:172.21.15.111:3300/0 conn(0x7f0c20162560 0x7f0c20072360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c1effd640 1 -- 172.21.15.111:0/2095161123 >> v1:172.21.15.52:6789/0 conn(0x7f0c2015e460 legacy=0x7f0c20161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:17:59.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c1effd640 1 -- 172.21.15.111:0/2095161123 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c20071880 con 0x7f0c2015e080 2024-05-04T12:17:59.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c1effd640 1 --2- 172.21.15.111:0/2095161123 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c2015e080 0x7f0c200b5cf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009870 tx=0x7f0c1401c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a69fbe25e8a20e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:17:59.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c1dffb640 1 -- 172.21.15.111:0/2095161123 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c140aa430 con 0x7f0c2015e080 2024-05-04T12:17:59.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c1dffb640 1 -- 172.21.15.111:0/2095161123 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c140af070 con 0x7f0c2015e080 2024-05-04T12:17:59.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c1dffb640 1 -- 172.21.15.111:0/2095161123 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c140aa7f0 con 0x7f0c2015e080 2024-05-04T12:17:59.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c252c0640 1 -- 172.21.15.111:0/2095161123 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c2015e080 msgr2=0x7f0c200b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c252c0640 1 --2- 172.21.15.111:0/2095161123 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c2015e080 0x7f0c200b5cf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009870 tx=0x7f0c1401c4f0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c252c0640 1 -- 172.21.15.111:0/2095161123 shutdown_connections 2024-05-04T12:17:59.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c252c0640 1 --2- 172.21.15.111:0/2095161123 >> v2:172.21.15.111:3300/0 conn(0x7f0c20162560 0x7f0c20072360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c252c0640 1 --2- 172.21.15.111:0/2095161123 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c2015e080 0x7f0c200b5cf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c252c0640 1 -- 172.21.15.111:0/2095161123 >> 172.21.15.111:0/2095161123 conn(0x7f0c2006b210 msgr2=0x7f0c2006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:17:59.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c252c0640 1 -- 172.21.15.111:0/2095161123 shutdown_connections 2024-05-04T12:17:59.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.811+0000 7f0c252c0640 1 -- 172.21.15.111:0/2095161123 wait complete. 2024-05-04T12:17:59.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c252c0640 1 Processor -- start 2024-05-04T12:17:59.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c252c0640 1 -- start start 2024-05-04T12:17:59.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c252c0640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c2015e080 0x7f0c20160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c252c0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c2015e460 0x7f0c2014f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c252c0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c20162560 0x7f0c2015f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:17:59.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c252c0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0c200734c0 con 0x7f0c2015e460 2024-05-04T12:17:59.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c252c0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0c20073340 con 0x7f0c20162560 2024-05-04T12:17:59.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c252c0640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f0c20073640 con 0x7f0c2015e080 2024-05-04T12:17:59.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c1effd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c2015e080 0x7f0c20160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c1f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c20162560 0x7f0c2015f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c1effd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c2015e080 0x7f0c20160f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:38170/0 (socket says 172.21.15.111:38170) 2024-05-04T12:17:59.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c1f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c20162560 0x7f0c2015f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43154/0 (socket says 172.21.15.111:43154) 2024-05-04T12:17:59.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c1effd640 1 -- 172.21.15.111:0/2326291949 learned_addr learned my addr 172.21.15.111:0/2326291949 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-04T12:17:59.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c1e7fc640 1 --2- 172.21.15.111:0/2326291949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c2015e460 0x7f0c2014f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c1f7fe640 1 -- 172.21.15.111:0/2326291949 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c2015e080 msgr2=0x7f0c20160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c1f7fe640 1 --2- 172.21.15.111:0/2326291949 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c2015e080 0x7f0c20160f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c1f7fe640 1 -- 172.21.15.111:0/2326291949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c2015e460 msgr2=0x7f0c2014f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c1f7fe640 1 --2- 172.21.15.111:0/2326291949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c2015e460 0x7f0c2014f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c1f7fe640 1 -- 172.21.15.111:0/2326291949 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c20161620 con 0x7f0c20162560 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c1effd640 1 --2- 172.21.15.111:0/2326291949 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c2015e080 0x7f0c20160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c1f7fe640 1 --2- 172.21.15.111:0/2326291949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c20162560 0x7f0c2015f440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800ebd0 tx=0x7f0c1800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c077fe640 1 -- 172.21.15.111:0/2326291949 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18018020 con 0x7f0c20162560 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c077fe640 1 -- 172.21.15.111:0/2326291949 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1800cd20 con 0x7f0c20162560 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c252c0640 1 -- 172.21.15.111:0/2326291949 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2014fde0 con 0x7f0c20162560 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c077fe640 1 -- 172.21.15.111:0/2326291949 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c180146c0 con 0x7f0c20162560 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c252c0640 1 -- 172.21.15.111:0/2326291949 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c2014f9b0 con 0x7f0c20162560 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c077fe640 1 -- 172.21.15.111:0/2326291949 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0c1801b020 con 0x7f0c20162560 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c077fe640 1 -- 172.21.15.111:0/2326291949 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f0c1801dd30 con 0x7f0c20162560 2024-05-04T12:17:59.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.815+0000 7f0c252c0640 1 -- 172.21.15.111:0/2326291949 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c20062e50 con 0x7f0c20162560 2024-05-04T12:17:59.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.819+0000 7f0c077fe640 1 -- 172.21.15.111:0/2326291949 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0c18022030 con 0x7f0c20162560 2024-05-04T12:17:59.859 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:17:59.859+0000 7fcc39d6e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-04T12:17:59.861 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:17:59.859+0000 7f5767e52180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-04T12:17:59.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.887+0000 7f0c252c0640 1 -- 172.21.15.111:0/2326291949 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7115a071-f397-4df5-b177-83637c288ff1", "id": 5} v 0) -- 0x7f0c20075320 con 0x7f0c20162560 2024-05-04T12:17:59.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.891+0000 7f0c077fe640 1 -- 172.21.15.111:0/2326291949 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7115a071-f397-4df5-b177-83637c288ff1", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f0c18010040 con 0x7f0c20162560 2024-05-04T12:17:59.893 INFO:teuthology.orchestra.run.smithi111.stdout:5 2024-05-04T12:17:59.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.895+0000 7f0c252c0640 1 -- 172.21.15.111:0/2326291949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c20162560 msgr2=0x7f0c2015f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:17:59.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.895+0000 7f0c252c0640 1 --2- 172.21.15.111:0/2326291949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c20162560 0x7f0c2015f440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800ebd0 tx=0x7f0c1800c490 comp rx=0 tx=0).stop 2024-05-04T12:17:59.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.895+0000 7f0c252c0640 1 -- 172.21.15.111:0/2326291949 shutdown_connections 2024-05-04T12:17:59.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.895+0000 7f0c252c0640 1 --2- 172.21.15.111:0/2326291949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c20162560 0x7f0c2015f440 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.895+0000 7f0c252c0640 1 --2- 172.21.15.111:0/2326291949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c2015e460 0x7f0c2014f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.895+0000 7f0c252c0640 1 --2- 172.21.15.111:0/2326291949 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c2015e080 0x7f0c20160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:17:59.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.895+0000 7f0c252c0640 1 -- 172.21.15.111:0/2326291949 >> 172.21.15.111:0/2326291949 conn(0x7f0c2006b210 msgr2=0x7f0c200b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:17:59.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.895+0000 7f0c252c0640 1 -- 172.21.15.111:0/2326291949 shutdown_connections 2024-05-04T12:17:59.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:17:59.895+0000 7f0c252c0640 1 -- 172.21.15.111:0/2326291949 wait complete. 2024-05-04T12:17:59.910 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new 35f3218b-c24f-44b6-81b0-dafaa4dc084e 6 2024-05-04T12:17:59.941 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:17:59.939+0000 7f5767e52180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-04T12:17:59.996 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:17:59.995+0000 7f5767e52180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-04T12:18:00.018 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:00.015+0000 7fcc39d6e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-04T12:18:00.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47f6c9640 1 Processor -- start 2024-05-04T12:18:00.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47f6c9640 1 -- start start 2024-05-04T12:18:00.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47f6c9640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4781531b0 0x7fb4780b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47f6c9640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb478160380 0x7fb4781607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47f6c9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4780cd360 con 0x7fb4780cce10 2024-05-04T12:18:00.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47f6c9640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb4780cdc00 con 0x7fb478160380 2024-05-04T12:18:00.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47f6c9640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fb4780cd640 con 0x7fb4781531b0 2024-05-04T12:18:00.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47cc64640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb4780cce10 0x7fb47815c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58004/0 (socket says 172.21.15.111:58004) 2024-05-04T12:18:00.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47cc64640 1 -- 172.21.15.111:0/2022893794 learned_addr learned my addr 172.21.15.111:0/2022893794 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-04T12:18:00.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47d465640 1 --2- 172.21.15.111:0/2022893794 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4781531b0 0x7fb4780b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47dc66640 1 --2- 172.21.15.111:0/2022893794 >> v2:172.21.15.52:3300/0 conn(0x7fb478160380 0x7fb4781607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb46ffff640 1 -- 172.21.15.111:0/2022893794 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1522627464 0 0) 0x7fb4780cd360 con 0x7fb4780cce10 2024-05-04T12:18:00.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb46ffff640 1 -- 172.21.15.111:0/2022893794 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb464003530 con 0x7fb4780cce10 2024-05-04T12:18:00.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47d465640 1 -- 172.21.15.111:0/2022893794 >> v2:172.21.15.52:3300/0 conn(0x7fb478160380 msgr2=0x7fb4781607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:00.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47d465640 1 --2- 172.21.15.111:0/2022893794 >> v2:172.21.15.52:3300/0 conn(0x7fb478160380 0x7fb4781607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47d465640 1 -- 172.21.15.111:0/2022893794 >> v1:172.21.15.111:6789/0 conn(0x7fb4780cce10 legacy=0x7fb47815c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.051+0000 7fb47d465640 1 -- 172.21.15.111:0/2022893794 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4780cd920 con 0x7fb4781531b0 2024-05-04T12:18:00.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.055+0000 7fb47d465640 1 --2- 172.21.15.111:0/2022893794 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4781531b0 0x7fb4780b4660 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb4740097c0 tx=0x7fb47401c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36b2f2e8e2082dc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:00.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.055+0000 7fb46ffff640 1 -- 172.21.15.111:0/2022893794 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4740b0070 con 0x7fb4781531b0 2024-05-04T12:18:00.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.055+0000 7fb46ffff640 1 -- 172.21.15.111:0/2022893794 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4740ab600 con 0x7fb4781531b0 2024-05-04T12:18:00.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.055+0000 7fb46ffff640 1 -- 172.21.15.111:0/2022893794 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4740a4e60 con 0x7fb4781531b0 2024-05-04T12:18:00.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 -- 172.21.15.111:0/2022893794 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4781531b0 msgr2=0x7fb4780b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 --2- 172.21.15.111:0/2022893794 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4781531b0 0x7fb4780b4660 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb4740097c0 tx=0x7fb47401c380 comp rx=0 tx=0).stop 2024-05-04T12:18:00.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 -- 172.21.15.111:0/2022893794 shutdown_connections 2024-05-04T12:18:00.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 --2- 172.21.15.111:0/2022893794 >> v2:172.21.15.52:3300/0 conn(0x7fb478160380 0x7fb4781607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 --2- 172.21.15.111:0/2022893794 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4781531b0 0x7fb4780b4660 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 -- 172.21.15.111:0/2022893794 >> 172.21.15.111:0/2022893794 conn(0x7fb478070690 msgr2=0x7fb478070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:00.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 -- 172.21.15.111:0/2022893794 shutdown_connections 2024-05-04T12:18:00.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 -- 172.21.15.111:0/2022893794 wait complete. 2024-05-04T12:18:00.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 Processor -- start 2024-05-04T12:18:00.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 -- start start 2024-05-04T12:18:00.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4780cce10 0x7fb4780aa330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4781531b0 0x7fb4781564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb478160380 0x7fb47814bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb478169250 con 0x7fb4780cce10 2024-05-04T12:18:00.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781690d0 con 0x7fb4781531b0 2024-05-04T12:18:00.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fb4781693d0 con 0x7fb478160380 2024-05-04T12:18:00.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47cc64640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4781531b0 0x7fb4781564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47dc66640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb478160380 0x7fb47814bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47cc64640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4781531b0 0x7fb4781564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43158/0 (socket says 172.21.15.111:43158) 2024-05-04T12:18:00.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47cc64640 1 -- 172.21.15.111:0/1487059244 learned_addr learned my addr 172.21.15.111:0/1487059244 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-04T12:18:00.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47d465640 1 --2- 172.21.15.111:0/1487059244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4780cce10 0x7fb4780aa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47cc64640 1 -- 172.21.15.111:0/1487059244 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb478160380 msgr2=0x7fb47814bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47cc64640 1 --2- 172.21.15.111:0/1487059244 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb478160380 0x7fb47814bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47cc64640 1 -- 172.21.15.111:0/1487059244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4780cce10 msgr2=0x7fb4780aa330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47cc64640 1 --2- 172.21.15.111:0/1487059244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4780cce10 0x7fb4780aa330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47cc64640 1 -- 172.21.15.111:0/1487059244 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb478156a00 con 0x7fb4781531b0 2024-05-04T12:18:00.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47cc64640 1 --2- 172.21.15.111:0/1487059244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4781531b0 0x7fb4781564c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb46800c3c0 tx=0x7fb46800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb46dffb640 1 -- 172.21.15.111:0/1487059244 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468017070 con 0x7fb4781531b0 2024-05-04T12:18:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb46dffb640 1 -- 172.21.15.111:0/1487059244 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb468005cf0 con 0x7fb4781531b0 2024-05-04T12:18:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 -- 172.21.15.111:0/1487059244 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47814a6f0 con 0x7fb4781531b0 2024-05-04T12:18:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb46dffb640 1 -- 172.21.15.111:0/1487059244 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4680072f0 con 0x7fb4781531b0 2024-05-04T12:18:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 -- 172.21.15.111:0/1487059244 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47814acb0 con 0x7fb4781531b0 2024-05-04T12:18:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb46dffb640 1 -- 172.21.15.111:0/1487059244 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb46801a070 con 0x7fb4781531b0 2024-05-04T12:18:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb46dffb640 1 -- 172.21.15.111:0/1487059244 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fb4680129f0 con 0x7fb4781531b0 2024-05-04T12:18:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.059+0000 7fb47f6c9640 1 -- 172.21.15.111:0/1487059244 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb478062e50 con 0x7fb4781531b0 2024-05-04T12:18:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.063+0000 7fb46dffb640 1 -- 172.21.15.111:0/1487059244 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb468021020 con 0x7fb4781531b0 2024-05-04T12:18:00.081 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:00.079+0000 7fcc39d6e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-04T12:18:00.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.131+0000 7fb47f6c9640 1 -- 172.21.15.111:0/1487059244 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd new", "uuid": "35f3218b-c24f-44b6-81b0-dafaa4dc084e", "id": 6} v 0) -- 0x7fb478075320 con 0x7fb4781531b0 2024-05-04T12:18:00.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.139+0000 7fb46dffb640 1 -- 172.21.15.111:0/1487059244 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "35f3218b-c24f-44b6-81b0-dafaa4dc084e", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fb478156a00 con 0x7fb4781531b0 2024-05-04T12:18:00.139 INFO:teuthology.orchestra.run.smithi111.stdout:6 2024-05-04T12:18:00.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.139+0000 7fb47f6c9640 1 -- 172.21.15.111:0/1487059244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4781531b0 msgr2=0x7fb4781564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.139+0000 7fb47f6c9640 1 --2- 172.21.15.111:0/1487059244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4781531b0 0x7fb4781564c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb46800c3c0 tx=0x7fb46800c880 comp rx=0 tx=0).stop 2024-05-04T12:18:00.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.139+0000 7fb47f6c9640 1 -- 172.21.15.111:0/1487059244 shutdown_connections 2024-05-04T12:18:00.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.139+0000 7fb47f6c9640 1 --2- 172.21.15.111:0/1487059244 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb478160380 0x7fb47814bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.139+0000 7fb47f6c9640 1 --2- 172.21.15.111:0/1487059244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4781531b0 0x7fb4781564c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.139+0000 7fb47f6c9640 1 --2- 172.21.15.111:0/1487059244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4780cce10 0x7fb4780aa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.139+0000 7fb47f6c9640 1 -- 172.21.15.111:0/1487059244 >> 172.21.15.111:0/1487059244 conn(0x7fb478070690 msgr2=0x7fb47814f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:00.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.139+0000 7fb47f6c9640 1 -- 172.21.15.111:0/1487059244 shutdown_connections 2024-05-04T12:18:00.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.139+0000 7fb47f6c9640 1 -- 172.21.15.111:0/1487059244 wait complete. 2024-05-04T12:18:00.154 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:18:00.155+0000 7f5767e52180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-04T12:18:00.156 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new 1a440ec5-ccd5-49ea-a894-7a03a742640b 7 2024-05-04T12:18:00.217 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:18:00.215+0000 7f5767e52180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-04T12:18:00.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.267+0000 7f3f3e0de640 1 Processor -- start 2024-05-04T12:18:00.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.267+0000 7f3f3e0de640 1 -- start start 2024-05-04T12:18:00.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.267+0000 7f3f3e0de640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3f38073370 0x7f3f380b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.267+0000 7f3f3e0de640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f38162c70 con 0x7f3f3815e080 2024-05-04T12:18:00.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.267+0000 7f3f3e0de640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f380cea90 con 0x7f3f3815e460 2024-05-04T12:18:00.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.267+0000 7f3f3e0de640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f3f38071ed0 con 0x7f3f38073370 2024-05-04T12:18:00.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.267+0000 7f3f36ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3f3815e460 0x7f3f38162560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58020/0 (socket says 172.21.15.111:58020) 2024-05-04T12:18:00.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.267+0000 7f3f36ffd640 1 -- 172.21.15.111:0/3053832836 learned_addr learned my addr 172.21.15.111:0/3053832836 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-04T12:18:00.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f37fff640 1 --2- 172.21.15.111:0/3053832836 >> v2:172.21.15.52:3300/0 conn(0x7f3f38073370 0x7f3f380b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 -- 172.21.15.111:0/3053832836 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1904644375 0 0) 0x7f3f380cea90 con 0x7f3f3815e460 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 -- 172.21.15.111:0/3053832836 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f20003530 con 0x7f3f3815e460 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 -- 172.21.15.111:0/3053832836 <== mon.1 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1507979884 0 0) 0x7f3f20003530 con 0x7f3f3815e460 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 -- 172.21.15.111:0/3053832836 >> v2:172.21.15.52:3300/0 conn(0x7f3f38073370 msgr2=0x7f3f380b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 --2- 172.21.15.111:0/3053832836 >> v2:172.21.15.52:3300/0 conn(0x7f3f38073370 0x7f3f380b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 -- 172.21.15.111:0/3053832836 >> v1:172.21.15.52:6789/0 conn(0x7f3f3815e080 legacy=0x7f3f38161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 -- 172.21.15.111:0/3053832836 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f380b6300 con 0x7f3f3815e460 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 -- 172.21.15.111:0/3053832836 <== mon.1 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f3f2c001680 con 0x7f3f3815e460 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f37fff640 1 --2- 172.21.15.111:0/3053832836 >> v2:172.21.15.52:3300/0 conn(0x7f3f38073370 0x7f3f380b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 -- 172.21.15.111:0/3053832836 >> v1:172.21.15.111:6789/0 conn(0x7f3f3815e460 legacy=0x7f3f38162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 --2- 172.21.15.111:0/3053832836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f200039c0 0x7f3f20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 --2- 172.21.15.111:0/3053832836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f20004810 0x7f3f20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 --2- 172.21.15.111:0/3053832836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f200051d0 0x7f3f2000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 -- 172.21.15.111:0/3053832836 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f38071ed0 con 0x7f3f200039c0 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 -- 172.21.15.111:0/3053832836 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2000c120 con 0x7f3f20004810 2024-05-04T12:18:00.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 -- 172.21.15.111:0/3053832836 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2000c320 con 0x7f3f200051d0 2024-05-04T12:18:00.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f36ffd640 1 --2- 172.21.15.111:0/3053832836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f20004810 0x7f3f20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f37fff640 1 --2- 172.21.15.111:0/3053832836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f200051d0 0x7f3f2000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f377fe640 1 --2- 172.21.15.111:0/3053832836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f200039c0 0x7f3f20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f36ffd640 1 -- 172.21.15.111:0/3053832836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f200051d0 msgr2=0x7f3f2000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:00.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f36ffd640 1 --2- 172.21.15.111:0/3053832836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f200051d0 0x7f3f2000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f36ffd640 1 -- 172.21.15.111:0/3053832836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f200039c0 msgr2=0x7f3f20003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:00.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f36ffd640 1 --2- 172.21.15.111:0/3053832836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f200039c0 0x7f3f20003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f36ffd640 1 -- 172.21.15.111:0/3053832836 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f20003530 con 0x7f3f20004810 2024-05-04T12:18:00.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f37fff640 1 -- 172.21.15.111:0/3053832836 reap_dead start 2024-05-04T12:18:00.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f36ffd640 1 --2- 172.21.15.111:0/3053832836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f20004810 0x7f3f20004c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c1c0 tx=0x7f3f2c01cbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d2d323a2fc5eaff server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:00.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 -- 172.21.15.111:0/3053832836 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c007d00 con 0x7f3f20004810 2024-05-04T12:18:00.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 -- 172.21.15.111:0/3053832836 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f2c0a9020 con 0x7f3f20004810 2024-05-04T12:18:00.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f367fc640 1 -- 172.21.15.111:0/3053832836 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c005ae0 con 0x7f3f20004810 2024-05-04T12:18:00.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f3e0de640 1 -- 172.21.15.111:0/3053832836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f20004810 msgr2=0x7f3f20004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f3e0de640 1 --2- 172.21.15.111:0/3053832836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f20004810 0x7f3f20004c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c1c0 tx=0x7f3f2c01cbb0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f3e0de640 1 -- 172.21.15.111:0/3053832836 shutdown_connections 2024-05-04T12:18:00.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f3e0de640 1 --2- 172.21.15.111:0/3053832836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f20004810 0x7f3f20004c90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f3e0de640 1 -- 172.21.15.111:0/3053832836 >> 172.21.15.111:0/3053832836 conn(0x7f3f3806b210 msgr2=0x7f3f3806b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:00.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f3e0de640 1 -- 172.21.15.111:0/3053832836 shutdown_connections 2024-05-04T12:18:00.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.271+0000 7f3f3e0de640 1 -- 172.21.15.111:0/3053832836 wait complete. 2024-05-04T12:18:00.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f3e0de640 1 Processor -- start 2024-05-04T12:18:00.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f3e0de640 1 -- start start 2024-05-04T12:18:00.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f3e0de640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f20004810 0x7f3f3814d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f3e0de640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3814dd70 0x7f3f3815fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f3e0de640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f3815fff0 0x7f3f380cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f3e0de640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f381627e0 con 0x7f3f3815fff0 2024-05-04T12:18:00.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f3e0de640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3f38162660 con 0x7f3f3814dd70 2024-05-04T12:18:00.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f3e0de640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f3f38162960 con 0x7f3f20004810 2024-05-04T12:18:00.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f377fe640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f20004810 0x7f3f3814d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f36ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3814dd70 0x7f3f3815fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f377fe640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f20004810 0x7f3f3814d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:38218/0 (socket says 172.21.15.111:38218) 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f377fe640 1 -- 172.21.15.111:0/3669482832 learned_addr learned my addr 172.21.15.111:0/3669482832 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f37fff640 1 --2- 172.21.15.111:0/3669482832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f3815fff0 0x7f3f380cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f36ffd640 1 -- 172.21.15.111:0/3669482832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f20004810 msgr2=0x7f3f3814d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f36ffd640 1 --2- 172.21.15.111:0/3669482832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f20004810 0x7f3f3814d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f36ffd640 1 -- 172.21.15.111:0/3669482832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f3815fff0 msgr2=0x7f3f380cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f36ffd640 1 --2- 172.21.15.111:0/3669482832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f3815fff0 0x7f3f380cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f36ffd640 1 -- 172.21.15.111:0/3669482832 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f38160620 con 0x7f3f3814dd70 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f37fff640 1 --2- 172.21.15.111:0/3669482832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f3815fff0 0x7f3f380cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f36ffd640 1 --2- 172.21.15.111:0/3669482832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3814dd70 0x7f3f3815fab0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3f2000c2f0 tx=0x7f3f20005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f1ffff640 1 -- 172.21.15.111:0/3669482832 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0b4070 con 0x7f3f3814dd70 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f1ffff640 1 -- 172.21.15.111:0/3669482832 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f2c0070d0 con 0x7f3f3814dd70 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f3e0de640 1 -- 172.21.15.111:0/3669482832 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f38160900 con 0x7f3f3814dd70 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f1ffff640 1 -- 172.21.15.111:0/3669482832 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c007d00 con 0x7f3f3814dd70 2024-05-04T12:18:00.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f3e0de640 1 -- 172.21.15.111:0/3669482832 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f38159f40 con 0x7f3f3814dd70 2024-05-04T12:18:00.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f1ffff640 1 -- 172.21.15.111:0/3669482832 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3f2c0a5770 con 0x7f3f3814dd70 2024-05-04T12:18:00.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f1ffff640 1 -- 172.21.15.111:0/3669482832 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f3f2c0aea90 con 0x7f3f3814dd70 2024-05-04T12:18:00.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.275+0000 7f3f3e0de640 1 -- 172.21.15.111:0/3669482832 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f38062e50 con 0x7f3f3814dd70 2024-05-04T12:18:00.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.279+0000 7f3f1ffff640 1 -- 172.21.15.111:0/3669482832 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3f2c0168f0 con 0x7f3f3814dd70 2024-05-04T12:18:00.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.347+0000 7f3f3e0de640 1 -- 172.21.15.111:0/3669482832 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1a440ec5-ccd5-49ea-a894-7a03a742640b", "id": 7} v 0) -- 0x7f3f38075320 con 0x7f3f3814dd70 2024-05-04T12:18:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.355+0000 7f3f1ffff640 1 -- 172.21.15.111:0/3669482832 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1a440ec5-ccd5-49ea-a894-7a03a742640b", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f3f2c0b8450 con 0x7f3f3814dd70 2024-05-04T12:18:00.357 INFO:teuthology.orchestra.run.smithi111.stdout:7 2024-05-04T12:18:00.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.359+0000 7f3f3e0de640 1 -- 172.21.15.111:0/3669482832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3814dd70 msgr2=0x7f3f3815fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.359+0000 7f3f3e0de640 1 --2- 172.21.15.111:0/3669482832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3814dd70 0x7f3f3815fab0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3f2000c2f0 tx=0x7f3f20005840 comp rx=0 tx=0).stop 2024-05-04T12:18:00.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.359+0000 7f3f3e0de640 1 -- 172.21.15.111:0/3669482832 shutdown_connections 2024-05-04T12:18:00.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.359+0000 7f3f3e0de640 1 --2- 172.21.15.111:0/3669482832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f3815fff0 0x7f3f380cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.359+0000 7f3f3e0de640 1 --2- 172.21.15.111:0/3669482832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3814dd70 0x7f3f3815fab0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.359+0000 7f3f3e0de640 1 --2- 172.21.15.111:0/3669482832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f20004810 0x7f3f3814d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.359+0000 7f3f3e0de640 1 -- 172.21.15.111:0/3669482832 >> 172.21.15.111:0/3669482832 conn(0x7f3f3806b210 msgr2=0x7f3f380b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:00.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.359+0000 7f3f3e0de640 1 -- 172.21.15.111:0/3669482832 shutdown_connections 2024-05-04T12:18:00.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:00.359+0000 7f3f3e0de640 1 -- 172.21.15.111:0/3669482832 wait complete. 2024-05-04T12:18:00.373 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-04T12:18:00.373 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-04T12:18:00.376 INFO:tasks.ceph.osd.0:Started 2024-05-04T12:18:00.376 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-04T12:18:00.376 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-04T12:18:00.381 INFO:tasks.ceph.osd.1:Started 2024-05-04T12:18:00.381 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-04T12:18:00.381 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-04T12:18:00.383 INFO:tasks.ceph.osd.2:Started 2024-05-04T12:18:00.384 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-04T12:18:00.384 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-04T12:18:00.386 INFO:tasks.ceph.osd.3:Started 2024-05-04T12:18:00.386 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-04T12:18:00.386 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-04T12:18:00.388 INFO:tasks.ceph.osd.4:Started 2024-05-04T12:18:00.389 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-04T12:18:00.389 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-04T12:18:00.391 INFO:tasks.ceph.osd.5:Started 2024-05-04T12:18:00.391 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-04T12:18:00.391 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-04T12:18:00.393 INFO:tasks.ceph.osd.6:Started 2024-05-04T12:18:00.393 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-04T12:18:00.393 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-04T12:18:00.395 INFO:tasks.ceph.osd.7:Started 2024-05-04T12:18:00.395 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:00.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbfba71640 1 Processor -- start 2024-05-04T12:18:00.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbfba71640 1 -- start start 2024-05-04T12:18:00.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbfba71640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41608d0 0x7fbbf415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbfba71640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbf406dd90 con 0x7fbbf4070930 2024-05-04T12:18:00.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbfba71640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbf406e000 con 0x7fbbf4070d10 2024-05-04T12:18:00.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbfba71640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf406e2e0 con 0x7fbbf41608d0 2024-05-04T12:18:00.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf980d640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fbbf4070930 0x7fbbf41601c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:59488/0 (socket says 172.21.15.52:59488) 2024-05-04T12:18:00.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf980d640 1 -- 172.21.15.52:0/3044955326 learned_addr learned my addr 172.21.15.52:0/3044955326 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:00.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbfa00e640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41608d0 0x7fbbf415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1596733943 0 0) 0x7fbbf406dd90 con 0x7fbbf4070930 2024-05-04T12:18:00.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbe0003530 con 0x7fbbf4070930 2024-05-04T12:18:00.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1532251202 0 0) 0x7fbbf406e000 con 0x7fbbf4070d10 2024-05-04T12:18:00.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbf406dd90 con 0x7fbbf4070d10 2024-05-04T12:18:00.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3895375876 0 0) 0x7fbbe0003530 con 0x7fbbf4070930 2024-05-04T12:18:00.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41608d0 msgr2=0x7fbbf415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:00.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41608d0 0x7fbbf415cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 >> v1:172.21.15.111:6789/0 conn(0x7fbbf4070d10 legacy=0x7fbbf415c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf406e5c0 con 0x7fbbf4070930 2024-05-04T12:18:00.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7fbbec001600 con 0x7fbbf4070930 2024-05-04T12:18:00.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbfa00e640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41608d0 0x7fbbf415cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:18:00.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 >> v1:172.21.15.52:6789/0 conn(0x7fbbf4070930 legacy=0x7fbbf41601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41608d0 0x7fbbe0003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbe0004110 0x7fbbe0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe0004d50 0x7fbbe00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf406e2e0 con 0x7fbbe0004110 2024-05-04T12:18:00.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe000bb90 con 0x7fbbe0004d50 2024-05-04T12:18:00.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe000bd90 con 0x7fbbf41608d0 2024-05-04T12:18:00.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf900c640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbe0004110 0x7fbbe0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf980d640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41608d0 0x7fbbe0003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbfa00e640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe0004d50 0x7fbbe00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf900c640 1 -- 172.21.15.52:0/3044955326 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41608d0 msgr2=0x7fbbe0003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:00.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf900c640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41608d0 0x7fbbe0003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf900c640 1 -- 172.21.15.52:0/3044955326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe0004d50 msgr2=0x7fbbe00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:00.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf900c640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe0004d50 0x7fbbe00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf900c640 1 -- 172.21.15.52:0/3044955326 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe0003530 con 0x7fbbe0004110 2024-05-04T12:18:00.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbfa00e640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe0004d50 0x7fbbe00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:00.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.563+0000 7fbbf980d640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41608d0 0x7fbbe0003b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:00.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.567+0000 7fbbf900c640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbe0004110 0x7fbbe0004810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40018d0 tx=0x7fbbe401c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e976ddeccfefed79 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:00.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.567+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4001520 con 0x7fbbe0004110 2024-05-04T12:18:00.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.567+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbe40050d0 con 0x7fbbe0004110 2024-05-04T12:18:00.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.567+0000 7fbbf880b640 1 -- 172.21.15.52:0/3044955326 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4003600 con 0x7fbbe0004110 2024-05-04T12:18:00.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.567+0000 7fbbfba71640 1 -- 172.21.15.52:0/3044955326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbe0004110 msgr2=0x7fbbe0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.567+0000 7fbbfba71640 1 --2- 172.21.15.52:0/3044955326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbe0004110 0x7fbbe0004810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40018d0 tx=0x7fbbe401c580 comp rx=0 tx=0).stop 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.567+0000 7fbbfa00e640 1 -- 172.21.15.52:0/3044955326 reap_dead start 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.567+0000 7fbbfba71640 1 -- 172.21.15.52:0/3044955326 shutdown_connections 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.567+0000 7fbbfba71640 1 -- 172.21.15.52:0/3044955326 >> 172.21.15.52:0/3044955326 conn(0x7fbbf4067070 msgr2=0x7fbbf4067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.567+0000 7fbbfba71640 1 -- 172.21.15.52:0/3044955326 shutdown_connections 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.567+0000 7fbbfba71640 1 -- 172.21.15.52:0/3044955326 wait complete. 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.567+0000 7fbbfba71640 1 Processor -- start 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.567+0000 7fbbfba71640 1 -- start start 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbfba71640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbe0004110 0x7fbbf41f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbfba71640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf4072b60 0x7fbbf4156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbfba71640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41566a0 0x7fbbf41400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbfba71640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf4167c30 con 0x7fbbe0004110 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbfba71640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf4167ab0 con 0x7fbbf4072b60 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbfba71640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf4167db0 con 0x7fbbf41566a0 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbf980d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbe0004110 0x7fbbf41f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbf980d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbe0004110 0x7fbbf41f7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51778/0 (socket says 172.21.15.52:51778) 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbf980d640 1 -- 172.21.15.52:0/1608076033 learned_addr learned my addr 172.21.15.52:0/1608076033 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbf900c640 1 --2- 172.21.15.52:0/1608076033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf4072b60 0x7fbbf4156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbfa00e640 1 --2- 172.21.15.52:0/1608076033 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41566a0 0x7fbbf41400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbf980d640 1 -- 172.21.15.52:0/1608076033 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41566a0 msgr2=0x7fbbf41400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbf980d640 1 --2- 172.21.15.52:0/1608076033 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41566a0 0x7fbbf41400c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbf980d640 1 -- 172.21.15.52:0/1608076033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf4072b60 msgr2=0x7fbbf4156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbf980d640 1 --2- 172.21.15.52:0/1608076033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf4072b60 0x7fbbf4156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbf980d640 1 -- 172.21.15.52:0/1608076033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf40717e0 con 0x7fbbe0004110 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbfa00e640 1 --2- 172.21.15.52:0/1608076033 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41566a0 0x7fbbf41400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbf980d640 1 --2- 172.21.15.52:0/1608076033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbe0004110 0x7fbbf41f7590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00e7b0 tx=0x7fbbec00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbea7fc640 1 -- 172.21.15.52:0/1608076033 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec00c660 con 0x7fbbe0004110 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbea7fc640 1 -- 172.21.15.52:0/1608076033 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbec018070 con 0x7fbbe0004110 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbfba71640 1 -- 172.21.15.52:0/1608076033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf4071ac0 con 0x7fbbe0004110 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbea7fc640 1 -- 172.21.15.52:0/1608076033 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec00ca40 con 0x7fbbe0004110 2024-05-04T12:18:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbfba71640 1 -- 172.21.15.52:0/1608076033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf4072080 con 0x7fbbe0004110 2024-05-04T12:18:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbea7fc640 1 -- 172.21.15.52:0/1608076033 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbbec00cbe0 con 0x7fbbe0004110 2024-05-04T12:18:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbea7fc640 1 -- 172.21.15.52:0/1608076033 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fbbec013b90 con 0x7fbbe0004110 2024-05-04T12:18:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.571+0000 7fbbfba71640 1 -- 172.21.15.52:0/1608076033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf4062e50 con 0x7fbbe0004110 2024-05-04T12:18:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.575+0000 7fbbea7fc640 1 -- 172.21.15.52:0/1608076033 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbbec0134c0 con 0x7fbbe0004110 2024-05-04T12:18:00.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.635+0000 7fbbfba71640 1 -- 172.21.15.52:0/1608076033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbbf40afde0 con 0x7fbbe0004110 2024-05-04T12:18:00.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.635+0000 7fbbea7fc640 1 -- 172.21.15.52:0/1608076033 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fbbec0136a0 con 0x7fbbe0004110 2024-05-04T12:18:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:00.635 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":10,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:00.351464+0000","last_up_change":"0.000000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:00.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.635+0000 7fbbfba71640 1 -- 172.21.15.52:0/1608076033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbe0004110 msgr2=0x7fbbf41f7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:00.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.635+0000 7fbbfba71640 1 --2- 172.21.15.52:0/1608076033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbe0004110 0x7fbbf41f7590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00e7b0 tx=0x7fbbec00ec70 comp rx=0 tx=0).stop 2024-05-04T12:18:00.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.635+0000 7fbbfba71640 1 -- 172.21.15.52:0/1608076033 shutdown_connections 2024-05-04T12:18:00.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.635+0000 7fbbfba71640 1 --2- 172.21.15.52:0/1608076033 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf41566a0 0x7fbbf41400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.635+0000 7fbbfba71640 1 --2- 172.21.15.52:0/1608076033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf4072b60 0x7fbbf4156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.635+0000 7fbbfba71640 1 --2- 172.21.15.52:0/1608076033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbe0004110 0x7fbbf41f7590 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:00.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.635+0000 7fbbfba71640 1 -- 172.21.15.52:0/1608076033 >> 172.21.15.52:0/1608076033 conn(0x7fbbf4067070 msgr2=0x7fbbf413e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:00.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.635+0000 7fbbfba71640 1 -- 172.21.15.52:0/1608076033 shutdown_connections 2024-05-04T12:18:00.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:00.635+0000 7fbbfba71640 1 -- 172.21.15.52:0/1608076033 wait complete. 2024-05-04T12:18:00.652 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-04T12:18:00.652 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-04T12:18:00.724 INFO:tasks.ceph.mgr.y.smithi052.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-04T12:18:00.724 INFO:tasks.ceph.mgr.y.smithi052.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-04T12:18:00.725 INFO:tasks.ceph.mgr.y.smithi052.stderr: from numpy import show_config as show_numpy_config 2024-05-04T12:18:00.781 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:00.783+0000 7fcc39d6e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-04T12:18:00.850 INFO:tasks.ceph.mgr.x.smithi111.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-04T12:18:00.850 INFO:tasks.ceph.mgr.x.smithi111.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-04T12:18:00.850 INFO:tasks.ceph.mgr.x.smithi111.stderr: from numpy import show_config as show_numpy_config 2024-05-04T12:18:00.925 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:18:00.923+0000 7f5767e52180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-04T12:18:00.948 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:00.947+0000 7fcc39d6e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-04T12:18:01.007 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T12:18:01.007+0000 7f14165d57c0 -1 Falling back to public interface 2024-05-04T12:18:01.008 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T12:18:01.007+0000 7f81dfb707c0 -1 Falling back to public interface 2024-05-04T12:18:01.008 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T12:18:01.007+0000 7fc54162b7c0 -1 Falling back to public interface 2024-05-04T12:18:01.012 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T12:18:01.011+0000 7fb7513d27c0 -1 Falling back to public interface 2024-05-04T12:18:01.019 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T12:18:01.019+0000 7f3c9d5e17c0 -1 Falling back to public interface 2024-05-04T12:18:01.034 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T12:18:01.035+0000 7f7b94a8f7c0 -1 Falling back to public interface 2024-05-04T12:18:01.042 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T12:18:01.043+0000 7fd3c2b247c0 -1 Falling back to public interface 2024-05-04T12:18:01.058 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T12:18:01.059+0000 7f179c21c7c0 -1 Falling back to public interface 2024-05-04T12:18:01.113 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:18:01.111+0000 7f5767e52180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-04T12:18:01.120 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:01.119+0000 7fcc39d6e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-04T12:18:01.191 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:01.191+0000 7fcc39d6e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-04T12:18:01.290 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:18:01.291+0000 7f5767e52180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-04T12:18:01.335 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:01.335+0000 7fcc39d6e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-04T12:18:01.360 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:18:01.359+0000 7f5767e52180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-04T12:18:01.457 DEBUG:teuthology.orchestra.run.smithi052:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:01.507 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:18:01.507+0000 7f5767e52180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-04T12:18:01.573 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.571+0000 7f3baead4640 1 Processor -- start 2024-05-04T12:18:01.573 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.571+0000 7f3baead4640 1 -- start start 2024-05-04T12:18:01.575 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3baead4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3ba815ac10 0x7f3ba806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:01.575 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3baead4640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ba80b51b0 0x7f3ba806e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:01.575 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3baead4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ba815e9a0 con 0x7f3ba806cc00 2024-05-04T12:18:01.575 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3baead4640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f3ba815ec80 con 0x7f3ba815ac10 2024-05-04T12:18:01.575 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3baead4640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba815ef60 con 0x7f3ba80b51b0 2024-05-04T12:18:01.576 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3bac870640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3ba815ac10 0x7f3ba806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:01.576 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3bad071640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ba80b51b0 0x7f3ba806e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:01.576 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3bac870640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3ba815ac10 0x7f3ba806c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:49994/0 (socket says 172.21.15.52:49994) 2024-05-04T12:18:01.576 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3bac870640 1 -- 172.21.15.52:0/1538282152 learned_addr learned my addr 172.21.15.52:0/1538282152 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:01.576 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3bac870640 1 -- 172.21.15.52:0/1538282152 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ba80b51b0 msgr2=0x7f3ba806e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:01.576 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3bac870640 1 --2- 172.21.15.52:0/1538282152 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ba80b51b0 0x7f3ba806e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:01.576 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3bac870640 1 -- 172.21.15.52:0/1538282152 >> v1:172.21.15.111:6789/0 conn(0x7f3ba806cc00 legacy=0x7f3ba80b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:01.576 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3bac870640 1 -- 172.21.15.52:0/1538282152 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba815f240 con 0x7f3ba815ac10 2024-05-04T12:18:01.576 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3bad071640 1 --2- 172.21.15.52:0/1538282152 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ba80b51b0 0x7f3ba806e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:18:01.578 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.575+0000 7f3bac870640 1 --2- 172.21.15.52:0/1538282152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ba815ac10 0x7f3ba806c6c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009870 tx=0x7f3b9c01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eb387ba1e5bc7af server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:01.578 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.579+0000 7f3ba77fe640 1 -- 172.21.15.52:0/1538282152 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0b0070 con 0x7f3ba815ac10 2024-05-04T12:18:01.578 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.579+0000 7f3ba77fe640 1 -- 172.21.15.52:0/1538282152 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b9c0a4e50 con 0x7f3ba815ac10 2024-05-04T12:18:01.578 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.579+0000 7f3ba77fe640 1 -- 172.21.15.52:0/1538282152 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0ab820 con 0x7f3ba815ac10 2024-05-04T12:18:01.579 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.579+0000 7f3baead4640 1 -- 172.21.15.52:0/1538282152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ba815ac10 msgr2=0x7f3ba806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:01.579 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.579+0000 7f3baead4640 1 --2- 172.21.15.52:0/1538282152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ba815ac10 0x7f3ba806c6c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009870 tx=0x7f3b9c01c4d0 comp rx=0 tx=0).stop 2024-05-04T12:18:01.579 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.579+0000 7f3baead4640 1 -- 172.21.15.52:0/1538282152 shutdown_connections 2024-05-04T12:18:01.579 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.579+0000 7f3baead4640 1 --2- 172.21.15.52:0/1538282152 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ba80b51b0 0x7f3ba806e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:01.579 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.579+0000 7f3baead4640 1 --2- 172.21.15.52:0/1538282152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ba815ac10 0x7f3ba806c6c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:01.579 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.579+0000 7f3baead4640 1 -- 172.21.15.52:0/1538282152 >> 172.21.15.52:0/1538282152 conn(0x7f3ba80b2a80 msgr2=0x7f3ba80b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:01.579 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.579+0000 7f3baead4640 1 -- 172.21.15.52:0/1538282152 shutdown_connections 2024-05-04T12:18:01.579 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.579+0000 7f3baead4640 1 -- 172.21.15.52:0/1538282152 wait complete. 2024-05-04T12:18:01.580 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.579+0000 7f3baead4640 1 Processor -- start 2024-05-04T12:18:01.581 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.579+0000 7f3baead4640 1 -- start start 2024-05-04T12:18:01.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3baead4640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ba806cc00 0x7f3ba8150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:01.584 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3baead4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ba80b51b0 0x7f3ba814dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:01.584 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3baead4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ba815ac10 0x7f3ba8147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:01.584 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3baead4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba806f8a0 con 0x7f3ba815ac10 2024-05-04T12:18:01.584 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3baead4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba806f720 con 0x7f3ba80b51b0 2024-05-04T12:18:01.584 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3baead4640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba806d030 con 0x7f3ba806cc00 2024-05-04T12:18:01.584 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3bad071640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ba815ac10 0x7f3ba8147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:01.584 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3bad071640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ba815ac10 0x7f3ba8147520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51784/0 (socket says 172.21.15.52:51784) 2024-05-04T12:18:01.584 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3bad071640 1 -- 172.21.15.52:0/3387909790 learned_addr learned my addr 172.21.15.52:0/3387909790 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:01.584 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3bac870640 1 --2- 172.21.15.52:0/3387909790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ba806cc00 0x7f3ba8150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:01.584 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3ba7fff640 1 --2- 172.21.15.52:0/3387909790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ba80b51b0 0x7f3ba814dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:01.584 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3bad071640 1 -- 172.21.15.52:0/3387909790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ba806cc00 msgr2=0x7f3ba8150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:01.584 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3bad071640 1 --2- 172.21.15.52:0/3387909790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ba806cc00 0x7f3ba8150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:01.584 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3bad071640 1 -- 172.21.15.52:0/3387909790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ba80b51b0 msgr2=0x7f3ba814dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:01.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3bad071640 1 --2- 172.21.15.52:0/3387909790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ba80b51b0 0x7f3ba814dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:01.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3bad071640 1 -- 172.21.15.52:0/3387909790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba814e160 con 0x7f3ba815ac10 2024-05-04T12:18:01.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3bad071640 1 --2- 172.21.15.52:0/3387909790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ba815ac10 0x7f3ba8147520 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009f00 tx=0x7f3ba000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:01.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3ba57fa640 1 -- 172.21.15.52:0/3387909790 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0018030 con 0x7f3ba815ac10 2024-05-04T12:18:01.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3ba57fa640 1 -- 172.21.15.52:0/3387909790 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba000ce50 con 0x7f3ba815ac10 2024-05-04T12:18:01.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3baead4640 1 -- 172.21.15.52:0/3387909790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba814ce60 con 0x7f3ba815ac10 2024-05-04T12:18:01.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3ba57fa640 1 -- 172.21.15.52:0/3387909790 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0003bc0 con 0x7f3ba815ac10 2024-05-04T12:18:01.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3baead4640 1 -- 172.21.15.52:0/3387909790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba814d420 con 0x7f3ba815ac10 2024-05-04T12:18:01.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3ba57fa640 1 -- 172.21.15.52:0/3387909790 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3ba001b070 con 0x7f3ba815ac10 2024-05-04T12:18:01.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3ba57fa640 1 -- 172.21.15.52:0/3387909790 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f3ba001dbd0 con 0x7f3ba815ac10 2024-05-04T12:18:01.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.583+0000 7f3baead4640 1 -- 172.21.15.52:0/3387909790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba8068ba0 con 0x7f3ba815ac10 2024-05-04T12:18:01.586 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.587+0000 7f3ba57fa640 1 -- 172.21.15.52:0/3387909790 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3ba0022070 con 0x7f3ba815ac10 2024-05-04T12:18:01.643 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.643+0000 7f3baead4640 1 -- 172.21.15.52:0/3387909790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ba806e560 con 0x7f3ba815ac10 2024-05-04T12:18:01.644 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.643+0000 7f3ba57fa640 1 -- 172.21.15.52:0/3387909790 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f3ba0010040 con 0x7f3ba815ac10 2024-05-04T12:18:01.644 INFO:teuthology.misc.health.smithi052.stdout: 2024-05-04T12:18:01.644 INFO:teuthology.misc.health.smithi052.stdout:{"epoch":10,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:00.351464+0000","last_up_change":"0.000000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:01.645 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.643+0000 7f3baead4640 1 -- 172.21.15.52:0/3387909790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ba815ac10 msgr2=0x7f3ba8147520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:01.645 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.643+0000 7f3baead4640 1 --2- 172.21.15.52:0/3387909790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ba815ac10 0x7f3ba8147520 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009f00 tx=0x7f3ba000c490 comp rx=0 tx=0).stop 2024-05-04T12:18:01.645 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.643+0000 7f3baead4640 1 -- 172.21.15.52:0/3387909790 shutdown_connections 2024-05-04T12:18:01.645 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.643+0000 7f3baead4640 1 --2- 172.21.15.52:0/3387909790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ba815ac10 0x7f3ba8147520 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:01.645 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.643+0000 7f3baead4640 1 --2- 172.21.15.52:0/3387909790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ba80b51b0 0x7f3ba814dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:01.645 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.643+0000 7f3baead4640 1 --2- 172.21.15.52:0/3387909790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ba806cc00 0x7f3ba8150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:01.645 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.643+0000 7f3baead4640 1 -- 172.21.15.52:0/3387909790 >> 172.21.15.52:0/3387909790 conn(0x7f3ba80b2a80 msgr2=0x7f3ba8148240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:01.645 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.643+0000 7f3baead4640 1 -- 172.21.15.52:0/3387909790 shutdown_connections 2024-05-04T12:18:01.646 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:01.643+0000 7f3baead4640 1 -- 172.21.15.52:0/3387909790 wait complete. 2024-05-04T12:18:01.659 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-04T12:18:01.988 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:01.987+0000 7fcc39d6e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-04T12:18:02.066 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:02.067+0000 7fcc39d6e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-04T12:18:02.211 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:18:02.211+0000 7f5767e52180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-04T12:18:02.256 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:02.255+0000 7fcc39d6e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-04T12:18:02.286 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:18:02.287+0000 7f5767e52180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-04T12:18:02.334 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:02.335+0000 7fcc39d6e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-04T12:18:02.466 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:18:02.467+0000 7f5767e52180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-04T12:18:02.539 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:18:02.539+0000 7f5767e52180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-04T12:18:02.640 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:02.639+0000 7fcc39d6e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-04T12:18:02.705 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:02.703+0000 7fcc39d6e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-04T12:18:02.833 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:18:02.831+0000 7f5767e52180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-04T12:18:02.896 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-05-04T12:18:02.895+0000 7f5767e52180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-04T12:18:03.744 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:03.743+0000 7fcbe11b9640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-04T12:18:03.754 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-04T12:18:03.755+0000 7fcbe11b9640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-04T12:18:04.837 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T12:18:04.835+0000 7f7b94a8f7c0 -1 osd.0 0 log_to_monitors true 2024-05-04T12:18:04.849 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T12:18:04.847+0000 7fb7513d27c0 -1 osd.4 0 log_to_monitors true 2024-05-04T12:18:04.888 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T12:18:04.887+0000 7fc54162b7c0 -1 osd.7 0 log_to_monitors true 2024-05-04T12:18:04.927 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T12:18:04.927+0000 7fd3c2b247c0 -1 osd.1 0 log_to_monitors true 2024-05-04T12:18:04.938 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T12:18:04.939+0000 7f14165d57c0 -1 osd.6 0 log_to_monitors true 2024-05-04T12:18:04.946 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T12:18:04.947+0000 7f3c9d5e17c0 -1 osd.2 0 log_to_monitors true 2024-05-04T12:18:04.966 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T12:18:04.967+0000 7f81dfb707c0 -1 osd.5 0 log_to_monitors true 2024-05-04T12:18:04.970 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T12:18:04.971+0000 7f179c21c7c0 -1 osd.3 0 log_to_monitors true 2024-05-04T12:18:08.247 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T12:18:08.246+0000 7f3c98d67640 -1 osd.2 0 waiting for initial osdmap 2024-05-04T12:18:08.271 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T12:18:08.270+0000 7f3c913ae640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T12:18:08.296 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T12:18:08.294+0000 7f7b91a4b640 -1 osd.0 0 waiting for initial osdmap 2024-05-04T12:18:08.323 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T12:18:08.322+0000 7f7b8885c640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T12:18:08.380 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T12:18:08.378+0000 7fd3bfae0640 -1 osd.1 0 waiting for initial osdmap 2024-05-04T12:18:08.394 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T12:18:08.394+0000 7fd3b68f1640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T12:18:08.420 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T12:18:08.418+0000 7f17979a2640 -1 osd.3 0 waiting for initial osdmap 2024-05-04T12:18:08.451 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T12:18:08.450+0000 7f178ffe9640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T12:18:08.464 DEBUG:teuthology.orchestra.run.smithi052:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:08.579 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f578dd17640 1 Processor -- start 2024-05-04T12:18:08.579 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f578dd17640 1 -- start start 2024-05-04T12:18:08.581 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f578dd17640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f578815c1d0 0x7f57880ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:08.581 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f578dd17640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57880c5a00 con 0x7f57880c84b0 2024-05-04T12:18:08.581 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f578dd17640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f578815c5b0 con 0x7f57880cfaf0 2024-05-04T12:18:08.582 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f578dd17640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f57880cf690 con 0x7f578815c1d0 2024-05-04T12:18:08.582 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f5786ffd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f57880c84b0 0x7f57880d0770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46844/0 (socket says 172.21.15.52:46844) 2024-05-04T12:18:08.582 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f5786ffd640 1 -- 172.21.15.52:0/1489164883 learned_addr learned my addr 172.21.15.52:0/1489164883 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:08.582 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57877fe640 1 --2- 172.21.15.52:0/1489164883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f578815c1d0 0x7f57880ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:08.582 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1292475754 0 0) 0x7f57880c5a00 con 0x7f57880c84b0 2024-05-04T12:18:08.582 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5770003530 con 0x7f57880c84b0 2024-05-04T12:18:08.582 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1147603450 0 0) 0x7f578815c5b0 con 0x7f57880cfaf0 2024-05-04T12:18:08.582 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57880c5a00 con 0x7f57880cfaf0 2024-05-04T12:18:08.582 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1497021300 0 0) 0x7f5770003530 con 0x7f57880c84b0 2024-05-04T12:18:08.582 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f578815c1d0 msgr2=0x7f57880ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:08.582 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 --2- 172.21.15.52:0/1489164883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f578815c1d0 0x7f57880ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:08.582 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 >> v1:172.21.15.111:6789/0 conn(0x7f57880cfaf0 legacy=0x7f5788165d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:08.582 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57880cf3b0 con 0x7f57880c84b0 2024-05-04T12:18:08.582 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f577c001680 con 0x7f57880c84b0 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 >> v1:172.21.15.52:6789/0 conn(0x7f57880c84b0 legacy=0x7f57880d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 --2- 172.21.15.52:0/1489164883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5770003b80 0x7f5770003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 --2- 172.21.15.52:0/1489164883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5770004810 0x7f5770004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 --2- 172.21.15.52:0/1489164883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57700051d0 0x7f577000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f57880cf690 con 0x7f5770003b80 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f577000bfc0 con 0x7f5770004810 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f577000c1c0 con 0x7f57700051d0 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57877fe640 1 --2- 172.21.15.52:0/1489164883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f578815c1d0 0x7f57880ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57877fe640 1 --2- 172.21.15.52:0/1489164883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5770003b80 0x7f5770003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f5786ffd640 1 --2- 172.21.15.52:0/1489164883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5770004810 0x7f5770004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f5787fff640 1 --2- 172.21.15.52:0/1489164883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57700051d0 0x7f577000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57877fe640 1 -- 172.21.15.52:0/1489164883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57700051d0 msgr2=0x7f577000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57877fe640 1 --2- 172.21.15.52:0/1489164883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57700051d0 0x7f577000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57877fe640 1 -- 172.21.15.52:0/1489164883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5770004810 msgr2=0x7f5770004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57877fe640 1 --2- 172.21.15.52:0/1489164883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5770004810 0x7f5770004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:08.583 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f57877fe640 1 -- 172.21.15.52:0/1489164883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5770003530 con 0x7f5770003b80 2024-05-04T12:18:08.584 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.578+0000 7f5786ffd640 1 --2- 172.21.15.52:0/1489164883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5770004810 0x7f5770004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:08.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.582+0000 7f57877fe640 1 --2- 172.21.15.52:0/1489164883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5770003b80 0x7f5770003f80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5788071960 tx=0x7f577801ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3078e98be278657 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:08.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.582+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57780097f0 con 0x7f5770003b80 2024-05-04T12:18:08.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.582+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5778004030 con 0x7f5770003b80 2024-05-04T12:18:08.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.582+0000 7f57867fc640 1 -- 172.21.15.52:0/1489164883 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778004400 con 0x7f5770003b80 2024-05-04T12:18:08.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.582+0000 7f578dd17640 1 -- 172.21.15.52:0/1489164883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5770003b80 msgr2=0x7f5770003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:08.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.582+0000 7f578dd17640 1 --2- 172.21.15.52:0/1489164883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5770003b80 0x7f5770003f80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5788071960 tx=0x7f577801ca80 comp rx=0 tx=0).stop 2024-05-04T12:18:08.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.582+0000 7f5787fff640 1 -- 172.21.15.52:0/1489164883 reap_dead start 2024-05-04T12:18:08.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.582+0000 7f578dd17640 1 -- 172.21.15.52:0/1489164883 shutdown_connections 2024-05-04T12:18:08.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.582+0000 7f578dd17640 1 -- 172.21.15.52:0/1489164883 >> 172.21.15.52:0/1489164883 conn(0x7f57880d1d10 msgr2=0x7f57880d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:08.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.582+0000 7f578dd17640 1 -- 172.21.15.52:0/1489164883 shutdown_connections 2024-05-04T12:18:08.585 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.582+0000 7f578dd17640 1 -- 172.21.15.52:0/1489164883 wait complete. 2024-05-04T12:18:08.587 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.582+0000 7f578dd17640 1 Processor -- start 2024-05-04T12:18:08.587 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.582+0000 7f578dd17640 1 -- start start 2024-05-04T12:18:08.588 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f578dd17640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5770003b80 0x7f578814de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:08.588 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f578dd17640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57880b6740 0x7f57880b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:08.588 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f578dd17640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57880b7100 0x7f57880c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:08.588 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f578dd17640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5788167dc0 con 0x7f5770003b80 2024-05-04T12:18:08.588 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f578dd17640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5788167c40 con 0x7f57880b7100 2024-05-04T12:18:08.588 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f578dd17640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5788167f40 con 0x7f57880b6740 2024-05-04T12:18:08.588 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f57877fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5770003b80 0x7f578814de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:08.588 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f57877fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5770003b80 0x7f578814de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39824/0 (socket says 172.21.15.52:39824) 2024-05-04T12:18:08.588 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f57877fe640 1 -- 172.21.15.52:0/4196028415 learned_addr learned my addr 172.21.15.52:0/4196028415 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:08.588 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f5787fff640 1 --2- 172.21.15.52:0/4196028415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57880b7100 0x7f57880c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:08.588 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f5786ffd640 1 --2- 172.21.15.52:0/4196028415 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57880b6740 0x7f57880b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:08.588 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f57877fe640 1 -- 172.21.15.52:0/4196028415 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57880b6740 msgr2=0x7f57880b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:08.588 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f57877fe640 1 --2- 172.21.15.52:0/4196028415 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57880b6740 0x7f57880b6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:08.589 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f57877fe640 1 -- 172.21.15.52:0/4196028415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57880b7100 msgr2=0x7f57880c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:08.589 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f57877fe640 1 --2- 172.21.15.52:0/4196028415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57880b7100 0x7f57880c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:08.589 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f57877fe640 1 -- 172.21.15.52:0/4196028415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578814e7e0 con 0x7f5770003b80 2024-05-04T12:18:08.589 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f57877fe640 1 --2- 172.21.15.52:0/4196028415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5770003b80 0x7f578814de30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f577801c8b0 tx=0x7f5770005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:08.589 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f576ffff640 1 -- 172.21.15.52:0/4196028415 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57780b2040 con 0x7f5770003b80 2024-05-04T12:18:08.589 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f576ffff640 1 -- 172.21.15.52:0/4196028415 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57780bb070 con 0x7f5770003b80 2024-05-04T12:18:08.589 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f578dd17640 1 -- 172.21.15.52:0/4196028415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57880b7930 con 0x7f5770003b80 2024-05-04T12:18:08.589 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f576ffff640 1 -- 172.21.15.52:0/4196028415 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778003aa0 con 0x7f5770003b80 2024-05-04T12:18:08.589 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f578dd17640 1 -- 172.21.15.52:0/4196028415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5788154560 con 0x7f5770003b80 2024-05-04T12:18:08.589 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f578dd17640 1 -- 172.21.15.52:0/4196028415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5788068ba0 con 0x7f5770003b80 2024-05-04T12:18:08.590 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f576ffff640 1 -- 172.21.15.52:0/4196028415 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f5778003c40 con 0x7f5770003b80 2024-05-04T12:18:08.590 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f576ffff640 1 --2- 172.21.15.52:0/4196028415 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5774067860 0x7f5774069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:08.590 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f576ffff640 1 -- 172.21.15.52:0/4196028415 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f577811e870 con 0x7f5770003b80 2024-05-04T12:18:08.590 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f5786ffd640 1 --2- 172.21.15.52:0/4196028415 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5774067860 0x7f5774069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:08.590 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.586+0000 7f5786ffd640 1 --2- 172.21.15.52:0/4196028415 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5774067860 0x7f5774069d20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57880cad10 tx=0x7f577c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:08.655 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.654+0000 7f576ffff640 1 -- 172.21.15.52:0/4196028415 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f57780c4020 con 0x7f5770003b80 2024-05-04T12:18:08.827 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.826+0000 7f578dd17640 1 -- 172.21.15.52:0/4196028415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f578815c5b0 con 0x7f5770003b80 2024-05-04T12:18:08.828 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.826+0000 7f576ffff640 1 -- 172.21.15.52:0/4196028415 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f578814e7e0 con 0x7f5770003b80 2024-05-04T12:18:08.828 INFO:teuthology.misc.health.smithi052.stdout: 2024-05-04T12:18:08.828 INFO:teuthology.misc.health.smithi052.stdout:{"epoch":12,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:06.752244+0000","last_up_change":"0.000000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:08.830 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.830+0000 7f578dd17640 1 -- 172.21.15.52:0/4196028415 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5774067860 msgr2=0x7f5774069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:08.830 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.830+0000 7f578dd17640 1 --2- 172.21.15.52:0/4196028415 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5774067860 0x7f5774069d20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57880cad10 tx=0x7f577c007040 comp rx=0 tx=0).stop 2024-05-04T12:18:08.831 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.830+0000 7f578dd17640 1 -- 172.21.15.52:0/4196028415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5770003b80 msgr2=0x7f578814de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:08.831 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.830+0000 7f578dd17640 1 --2- 172.21.15.52:0/4196028415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5770003b80 0x7f578814de30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f577801c8b0 tx=0x7f5770005840 comp rx=0 tx=0).stop 2024-05-04T12:18:08.831 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.830+0000 7f578dd17640 1 -- 172.21.15.52:0/4196028415 shutdown_connections 2024-05-04T12:18:08.831 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.830+0000 7f578dd17640 1 --2- 172.21.15.52:0/4196028415 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5774067860 0x7f5774069d20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:08.831 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.830+0000 7f578dd17640 1 --2- 172.21.15.52:0/4196028415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57880b7100 0x7f57880c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:08.831 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.830+0000 7f578dd17640 1 --2- 172.21.15.52:0/4196028415 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57880b6740 0x7f57880b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:08.831 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.830+0000 7f578dd17640 1 --2- 172.21.15.52:0/4196028415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5770003b80 0x7f578814de30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:08.831 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.830+0000 7f578dd17640 1 -- 172.21.15.52:0/4196028415 >> 172.21.15.52:0/4196028415 conn(0x7f57880d1d10 msgr2=0x7f578815c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:08.831 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.830+0000 7f578dd17640 1 -- 172.21.15.52:0/4196028415 shutdown_connections 2024-05-04T12:18:08.831 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:08.830+0000 7f578dd17640 1 -- 172.21.15.52:0/4196028415 wait complete. 2024-05-04T12:18:08.844 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-04T12:18:08.883 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T12:18:08.883+0000 7f81db2f6640 -1 osd.5 0 waiting for initial osdmap 2024-05-04T12:18:08.898 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T12:18:08.899+0000 7fc53cdb1640 -1 osd.7 0 waiting for initial osdmap 2024-05-04T12:18:08.906 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T12:18:08.907+0000 7f81d393d640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T12:18:08.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T12:18:08.919+0000 7fb74cb58640 -1 osd.4 0 waiting for initial osdmap 2024-05-04T12:18:08.927 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T12:18:08.927+0000 7fc5353f8640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T12:18:08.934 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T12:18:08.935+0000 7f1411d5b640 -1 osd.6 0 waiting for initial osdmap 2024-05-04T12:18:08.951 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T12:18:08.951+0000 7fb74519f640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T12:18:08.958 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T12:18:08.959+0000 7f140a3a2640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T12:18:15.651 DEBUG:teuthology.orchestra.run.smithi052:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:15.775 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5dfe7c5640 1 Processor -- start 2024-05-04T12:18:15.775 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5dfe7c5640 1 -- start start 2024-05-04T12:18:15.777 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5dfe7c5640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5df806b210 0x7f5df806b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:15.777 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5dfe7c5640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5df80ad130 0x7f5df80ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:15.777 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5dfe7c5640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df815fa20 con 0x7f5df815f640 2024-05-04T12:18:15.777 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5dfe7c5640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f5df80734c0 con 0x7f5df80ad130 2024-05-04T12:18:15.777 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5dfe7c5640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5df8067620 con 0x7f5df806b210 2024-05-04T12:18:15.777 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5df7fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f5df815f640 0x7f5df80728a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46858/0 (socket says 172.21.15.52:46858) 2024-05-04T12:18:15.777 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5df7fff640 1 -- 172.21.15.52:0/1783880780 learned_addr learned my addr 172.21.15.52:0/1783880780 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:15.777 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5dfcd62640 1 --2- 172.21.15.52:0/1783880780 >> v2:172.21.15.52:3300/0 conn(0x7f5df80ad130 0x7f5df80ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:15.777 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5df77fe640 1 --2- 172.21.15.52:0/1783880780 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5df806b210 0x7f5df806b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:15.777 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5df6ffd640 1 -- 172.21.15.52:0/1783880780 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4165756331 0 0) 0x7f5de80014a0 con 0x7f5df815f640 2024-05-04T12:18:15.777 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5df6ffd640 1 -- 172.21.15.52:0/1783880780 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5de0003530 con 0x7f5df815f640 2024-05-04T12:18:15.778 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5dfcd62640 1 -- 172.21.15.52:0/1783880780 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5df806b210 msgr2=0x7f5df806b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:15.778 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5dfcd62640 1 --2- 172.21.15.52:0/1783880780 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5df806b210 0x7f5df806b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:15.778 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5dfcd62640 1 -- 172.21.15.52:0/1783880780 >> v1:172.21.15.52:6789/0 conn(0x7f5df815f640 legacy=0x7f5df80728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:15.778 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.774+0000 7f5dfcd62640 1 -- 172.21.15.52:0/1783880780 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df8067060 con 0x7f5df80ad130 2024-05-04T12:18:15.779 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.778+0000 7f5dfcd62640 1 --2- 172.21.15.52:0/1783880780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df80ad130 0x7f5df80ad5c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5df00097f0 tx=0x7f5df001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5e7bf00cbf15e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:15.779 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.778+0000 7f5df6ffd640 1 -- 172.21.15.52:0/1783880780 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00b0070 con 0x7f5df80ad130 2024-05-04T12:18:15.779 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.778+0000 7f5df6ffd640 1 -- 172.21.15.52:0/1783880780 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df00ab600 con 0x7f5df80ad130 2024-05-04T12:18:15.780 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.778+0000 7f5df6ffd640 1 -- 172.21.15.52:0/1783880780 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00ab9d0 con 0x7f5df80ad130 2024-05-04T12:18:15.780 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.778+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/1783880780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df80ad130 msgr2=0x7f5df80ad5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:15.780 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.778+0000 7f5dfe7c5640 1 --2- 172.21.15.52:0/1783880780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df80ad130 0x7f5df80ad5c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5df00097f0 tx=0x7f5df001c4a0 comp rx=0 tx=0).stop 2024-05-04T12:18:15.780 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.778+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/1783880780 shutdown_connections 2024-05-04T12:18:15.780 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.778+0000 7f5dfe7c5640 1 --2- 172.21.15.52:0/1783880780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df80ad130 0x7f5df80ad5c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:15.780 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.778+0000 7f5dfe7c5640 1 --2- 172.21.15.52:0/1783880780 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5df806b210 0x7f5df806b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:15.781 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.778+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/1783880780 >> 172.21.15.52:0/1783880780 conn(0x7f5df80b3310 msgr2=0x7f5df80b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:15.781 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.778+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/1783880780 shutdown_connections 2024-05-04T12:18:15.781 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.778+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/1783880780 wait complete. 2024-05-04T12:18:15.781 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.778+0000 7f5dfe7c5640 1 Processor -- start 2024-05-04T12:18:15.781 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.778+0000 7f5dfe7c5640 1 -- start start 2024-05-04T12:18:15.784 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5dfe7c5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df806b210 0x7f5df8158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:15.784 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5dfe7c5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df80ad130 0x7f5df8159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:15.784 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5dfe7c5640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5df815f640 0x7f5df8099200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:15.784 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5dfe7c5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5df8168cc0 con 0x7f5df806b210 2024-05-04T12:18:15.784 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5dfe7c5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5df80b1cf0 con 0x7f5df80ad130 2024-05-04T12:18:15.784 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5dfe7c5640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5df8168e40 con 0x7f5df815f640 2024-05-04T12:18:15.784 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df806b210 0x7f5df8158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:15.784 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df806b210 0x7f5df8158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39852/0 (socket says 172.21.15.52:39852) 2024-05-04T12:18:15.784 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df7fff640 1 -- 172.21.15.52:0/2138454030 learned_addr learned my addr 172.21.15.52:0/2138454030 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:15.784 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5dfcd62640 1 --2- 172.21.15.52:0/2138454030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5df815f640 0x7f5df8099200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:15.784 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df77fe640 1 --2- 172.21.15.52:0/2138454030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df80ad130 0x7f5df8159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:15.785 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df7fff640 1 -- 172.21.15.52:0/2138454030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5df815f640 msgr2=0x7f5df8099200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:15.785 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df7fff640 1 --2- 172.21.15.52:0/2138454030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5df815f640 0x7f5df8099200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:15.785 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df7fff640 1 -- 172.21.15.52:0/2138454030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df80ad130 msgr2=0x7f5df8159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:15.785 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df7fff640 1 --2- 172.21.15.52:0/2138454030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df80ad130 0x7f5df8159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:15.785 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df7fff640 1 -- 172.21.15.52:0/2138454030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df80ce1e0 con 0x7f5df806b210 2024-05-04T12:18:15.785 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df7fff640 1 --2- 172.21.15.52:0/2138454030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df806b210 0x7f5df8158e40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5de800cb50 tx=0x7f5de800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:15.785 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df4ff9640 1 -- 172.21.15.52:0/2138454030 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8017070 con 0x7f5df806b210 2024-05-04T12:18:15.785 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df4ff9640 1 -- 172.21.15.52:0/2138454030 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de800ad40 con 0x7f5df806b210 2024-05-04T12:18:15.785 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/2138454030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df80ce4c0 con 0x7f5df806b210 2024-05-04T12:18:15.785 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df4ff9640 1 -- 172.21.15.52:0/2138454030 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8012640 con 0x7f5df806b210 2024-05-04T12:18:15.785 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/2138454030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df80cea80 con 0x7f5df806b210 2024-05-04T12:18:15.785 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/2138454030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df80cec80 con 0x7f5df806b210 2024-05-04T12:18:15.786 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df4ff9640 1 -- 172.21.15.52:0/2138454030 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5de801a070 con 0x7f5df806b210 2024-05-04T12:18:15.786 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df4ff9640 1 --2- 172.21.15.52:0/2138454030 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5de40678f0 0x7f5de4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:15.786 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df4ff9640 1 -- 172.21.15.52:0/2138454030 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f5de8085740 con 0x7f5df806b210 2024-05-04T12:18:15.786 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df77fe640 1 --2- 172.21.15.52:0/2138454030 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5de40678f0 0x7f5de4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:15.786 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.782+0000 7f5df77fe640 1 --2- 172.21.15.52:0/2138454030 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5de40678f0 0x7f5de4069db0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5df80cda10 tx=0x7f5dec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:15.790 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.786+0000 7f5df4ff9640 1 -- 172.21.15.52:0/2138454030 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5df80cec80 con 0x7f5df806b210 2024-05-04T12:18:15.953 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.950+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/2138454030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5df81534e0 con 0x7f5df806b210 2024-05-04T12:18:15.954 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.954+0000 7f5df4ff9640 1 -- 172.21.15.52:0/2138454030 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7f5df815fa20 con 0x7f5df806b210 2024-05-04T12:18:15.955 INFO:teuthology.misc.health.smithi052.stdout: 2024-05-04T12:18:15.955 INFO:teuthology.misc.health.smithi052.stdout:{"epoch":17,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:13.269413+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:15.957 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.954+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/2138454030 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5de40678f0 msgr2=0x7f5de4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:15.957 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.954+0000 7f5dfe7c5640 1 --2- 172.21.15.52:0/2138454030 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5de40678f0 0x7f5de4069db0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5df80cda10 tx=0x7f5dec008040 comp rx=0 tx=0).stop 2024-05-04T12:18:15.957 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.954+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/2138454030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df806b210 msgr2=0x7f5df8158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:15.957 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.954+0000 7f5dfe7c5640 1 --2- 172.21.15.52:0/2138454030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df806b210 0x7f5df8158e40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5de800cb50 tx=0x7f5de800a430 comp rx=0 tx=0).stop 2024-05-04T12:18:15.957 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.954+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/2138454030 shutdown_connections 2024-05-04T12:18:15.957 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.954+0000 7f5dfe7c5640 1 --2- 172.21.15.52:0/2138454030 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5de40678f0 0x7f5de4069db0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:15.957 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.954+0000 7f5dfe7c5640 1 --2- 172.21.15.52:0/2138454030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5df815f640 0x7f5df8099200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:15.957 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.954+0000 7f5dfe7c5640 1 --2- 172.21.15.52:0/2138454030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df80ad130 0x7f5df8159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:15.957 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.954+0000 7f5dfe7c5640 1 --2- 172.21.15.52:0/2138454030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df806b210 0x7f5df8158e40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:15.957 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.954+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/2138454030 >> 172.21.15.52:0/2138454030 conn(0x7f5df80b3310 msgr2=0x7f5df806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:15.958 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.954+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/2138454030 shutdown_connections 2024-05-04T12:18:15.958 INFO:teuthology.misc.health.smithi052.stderr:2024-05-04T12:18:15.954+0000 7f5dfe7c5640 1 -- 172.21.15.52:0/2138454030 wait complete. 2024-05-04T12:18:15.973 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-04T12:18:15.973 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-04T12:18:15.973 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-04T12:18:15.973 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-04T12:18:15.975 INFO:tasks.ceph.mds.a:Started 2024-05-04T12:18:15.976 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-04T12:18:15.976 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-04T12:18:15.978 INFO:tasks.ceph.mds.c:Started 2024-05-04T12:18:15.978 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-04T12:18:15.978 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-04T12:18:15.981 INFO:tasks.ceph.mds.b:Started 2024-05-04T12:18:15.981 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-04T12:18:15.981 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-04T12:18:15.984 INFO:tasks.ceph.mds.d:Started 2024-05-04T12:18:15.984 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-04T12:18:15.984 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:16.051 INFO:tasks.ceph.mds.c.smithi052.stdout:starting mds.c at 2024-05-04T12:18:16.051 INFO:tasks.ceph.mds.a.smithi052.stdout:starting mds.a at 2024-05-04T12:18:16.081 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2024-05-04T12:18:16.085 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2024-05-04T12:18:16.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.110+0000 7f404af7d640 1 Processor -- start 2024-05-04T12:18:16.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.110+0000 7f404af7d640 1 -- start start 2024-05-04T12:18:16.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.110+0000 7f404af7d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f40440a9480 0x7f4044142170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.110+0000 7f404af7d640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f404413f6d0 con 0x7f4044160220 2024-05-04T12:18:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.110+0000 7f404af7d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40440a9910 con 0x7f40440d0480 2024-05-04T12:18:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.110+0000 7f404af7d640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f40441426b0 con 0x7f40440a9480 2024-05-04T12:18:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f4043fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f40440a9480 0x7f4044142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f404951a640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f4044160220 0x7f40440d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46862/0 (socket says 172.21.15.52:46862) 2024-05-04T12:18:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f4048d19640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f40440d0480 0x7f404415fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.52:39148/0 (socket says 172.21.15.52:39148) 2024-05-04T12:18:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f404951a640 1 -- 172.21.15.52:0/2052359325 learned_addr learned my addr 172.21.15.52:0/2052359325 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f4043fff640 1 -- 172.21.15.52:0/2052359325 >> v1:172.21.15.111:6789/0 conn(0x7f40440d0480 legacy=0x7f404415fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f40437fe640 1 -- 172.21.15.52:0/2052359325 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2681770169 0 0) 0x7f40440a9910 con 0x7f40440d0480 2024-05-04T12:18:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f4043fff640 1 -- 172.21.15.52:0/2052359325 >> v1:172.21.15.52:6789/0 conn(0x7f4044160220 legacy=0x7f40440d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f4043fff640 1 -- 172.21.15.52:0/2052359325 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40440a9bf0 con 0x7f40440a9480 2024-05-04T12:18:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f4043fff640 1 --2- 172.21.15.52:0/2052359325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40440a9480 0x7f4044142170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f40340097c0 tx=0x7f403401c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f561af0feaf7b3c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f40437fe640 1 -- 172.21.15.52:0/2052359325 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40340b0070 con 0x7f40440a9480 2024-05-04T12:18:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f40437fe640 1 -- 172.21.15.52:0/2052359325 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40340ab600 con 0x7f40440a9480 2024-05-04T12:18:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f40437fe640 1 -- 172.21.15.52:0/2052359325 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40340ab9d0 con 0x7f40440a9480 2024-05-04T12:18:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f404af7d640 1 -- 172.21.15.52:0/2052359325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40440a9480 msgr2=0x7f4044142170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f404af7d640 1 --2- 172.21.15.52:0/2052359325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40440a9480 0x7f4044142170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f40340097c0 tx=0x7f403401c2a0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f404af7d640 1 -- 172.21.15.52:0/2052359325 shutdown_connections 2024-05-04T12:18:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f404af7d640 1 --2- 172.21.15.52:0/2052359325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40440a9480 0x7f4044142170 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f404af7d640 1 -- 172.21.15.52:0/2052359325 >> 172.21.15.52:0/2052359325 conn(0x7f4044070690 msgr2=0x7f4044070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f404af7d640 1 -- 172.21.15.52:0/2052359325 shutdown_connections 2024-05-04T12:18:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.114+0000 7f404af7d640 1 -- 172.21.15.52:0/2052359325 wait complete. 2024-05-04T12:18:16.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f404af7d640 1 Processor -- start 2024-05-04T12:18:16.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f404af7d640 1 -- start start 2024-05-04T12:18:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f404af7d640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40440d0480 0x7f4044138a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f404af7d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4044160220 0x7f40441fb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f404af7d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f404415f250 0x7f4044155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f404af7d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4044169ec0 con 0x7f4044160220 2024-05-04T12:18:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f404af7d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4044160900 con 0x7f404415f250 2024-05-04T12:18:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f404af7d640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f404416a040 con 0x7f40440d0480 2024-05-04T12:18:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f4043fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4044160220 0x7f40441fb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f4043fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4044160220 0x7f40441fb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39882/0 (socket says 172.21.15.52:39882) 2024-05-04T12:18:16.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f4043fff640 1 -- 172.21.15.52:0/641266631 learned_addr learned my addr 172.21.15.52:0/641266631 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:16.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f404951a640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f404415f250 0x7f4044155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f4048d19640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40440d0480 0x7f4044138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f4043fff640 1 -- 172.21.15.52:0/641266631 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40440d0480 msgr2=0x7f4044138a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f4043fff640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40440d0480 0x7f4044138a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f4043fff640 1 -- 172.21.15.52:0/641266631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f404415f250 msgr2=0x7f4044155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f4043fff640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f404415f250 0x7f4044155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f4043fff640 1 -- 172.21.15.52:0/641266631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404415f7f0 con 0x7f4044160220 2024-05-04T12:18:16.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f404951a640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f404415f250 0x7f4044155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:16.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f4043fff640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4044160220 0x7f40441fb870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f40340097c0 tx=0x7f4034019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:16.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f40417fa640 1 -- 172.21.15.52:0/641266631 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40340b8030 con 0x7f4044160220 2024-05-04T12:18:16.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f40417fa640 1 -- 172.21.15.52:0/641266631 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40340b4410 con 0x7f4044160220 2024-05-04T12:18:16.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f40417fa640 1 -- 172.21.15.52:0/641266631 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40340b0040 con 0x7f4044160220 2024-05-04T12:18:16.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f404af7d640 1 -- 172.21.15.52:0/641266631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4044137f60 con 0x7f4044160220 2024-05-04T12:18:16.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f404af7d640 1 -- 172.21.15.52:0/641266631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4044138520 con 0x7f4044160220 2024-05-04T12:18:16.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.118+0000 7f404af7d640 1 -- 172.21.15.52:0/641266631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4044062e50 con 0x7f4044160220 2024-05-04T12:18:16.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.122+0000 7f40417fa640 1 -- 172.21.15.52:0/641266631 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f40340bb070 con 0x7f4044160220 2024-05-04T12:18:16.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.122+0000 7f40417fa640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f40240678c0 0x7f4024069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.122+0000 7f40417fa640 1 -- 172.21.15.52:0/641266631 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f403411f5f0 con 0x7f4044160220 2024-05-04T12:18:16.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.122+0000 7f4048d19640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f40240678c0 0x7f4024069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.122+0000 7f4048d19640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f40240678c0 0x7f4024069d80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f402c000f70 tx=0x7f402c007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:16.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.126+0000 7f40417fa640 1 -- 172.21.15.52:0/641266631 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4034015fb0 con 0x7f4044160220 2024-05-04T12:18:16.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.286+0000 7f404af7d640 1 -- 172.21.15.52:0/641266631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4044167c90 con 0x7f4044160220 2024-05-04T12:18:16.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.290+0000 7f40417fa640 1 -- 172.21.15.52:0/641266631 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7f40340f0470 con 0x7f4044160220 2024-05-04T12:18:16.294 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:16.294 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:13.269413+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:16.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.294+0000 7f404af7d640 1 -- 172.21.15.52:0/641266631 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f40240678c0 msgr2=0x7f4024069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.294+0000 7f404af7d640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f40240678c0 0x7f4024069d80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f402c000f70 tx=0x7f402c007b60 comp rx=0 tx=0).stop 2024-05-04T12:18:16.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.294+0000 7f404af7d640 1 -- 172.21.15.52:0/641266631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4044160220 msgr2=0x7f40441fb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.294+0000 7f404af7d640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4044160220 0x7f40441fb870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f40340097c0 tx=0x7f4034019040 comp rx=0 tx=0).stop 2024-05-04T12:18:16.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.294+0000 7f404af7d640 1 -- 172.21.15.52:0/641266631 shutdown_connections 2024-05-04T12:18:16.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.294+0000 7f404af7d640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f40240678c0 0x7f4024069d80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.294+0000 7f404af7d640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f404415f250 0x7f4044155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.294+0000 7f404af7d640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4044160220 0x7f40441fb870 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.294+0000 7f404af7d640 1 --2- 172.21.15.52:0/641266631 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40440d0480 0x7f4044138a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.294+0000 7f404af7d640 1 -- 172.21.15.52:0/641266631 >> 172.21.15.52:0/641266631 conn(0x7f4044070690 msgr2=0x7f4044155dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:16.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.294+0000 7f404af7d640 1 -- 172.21.15.52:0/641266631 shutdown_connections 2024-05-04T12:18:16.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.294+0000 7f404af7d640 1 -- 172.21.15.52:0/641266631 wait complete. 2024-05-04T12:18:16.313 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:18:16.314 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:18:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24f18b640 1 Processor -- start 2024-05-04T12:18:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24f18b640 1 -- start start 2024-05-04T12:18:16.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24f18b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe248141010 0x7fe2480d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24f18b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe248144570 0x7fe248143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24f18b640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2480d29c0 con 0x7fe2480d2540 2024-05-04T12:18:16.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24f18b640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fe2480d2ca0 con 0x7fe248141010 2024-05-04T12:18:16.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24f18b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fe2480d2f80 con 0x7fe248144570 2024-05-04T12:18:16.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24cf27640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe248141010 0x7fe2480d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24d728640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fe2480d2540 0x7fe248161db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58842/0 (socket says 172.21.15.52:58842) 2024-05-04T12:18:16.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24d728640 1 -- 172.21.15.52:0/2950291400 learned_addr learned my addr 172.21.15.52:0/2950291400 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:16.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24cf27640 1 -- 172.21.15.52:0/2950291400 >> v2:172.21.15.111:3300/0 conn(0x7fe248144570 msgr2=0x7fe248143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:16.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24cf27640 1 --2- 172.21.15.52:0/2950291400 >> v2:172.21.15.111:3300/0 conn(0x7fe248144570 0x7fe248143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24cf27640 1 -- 172.21.15.52:0/2950291400 >> v1:172.21.15.52:6789/0 conn(0x7fe2480d2540 legacy=0x7fe248161db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24cf27640 1 -- 172.21.15.52:0/2950291400 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2480d3260 con 0x7fe248141010 2024-05-04T12:18:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.434+0000 7fe24cf27640 1 --2- 172.21.15.52:0/2950291400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe248141010 0x7fe2480d2000 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe2480b3820 tx=0x7fe23801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1383dcc41e424d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe2477fe640 1 -- 172.21.15.52:0/2950291400 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2380b0070 con 0x7fe248141010 2024-05-04T12:18:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe2477fe640 1 -- 172.21.15.52:0/2950291400 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2380ab600 con 0x7fe248141010 2024-05-04T12:18:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe2477fe640 1 -- 172.21.15.52:0/2950291400 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2380ab9d0 con 0x7fe248141010 2024-05-04T12:18:16.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 -- 172.21.15.52:0/2950291400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe248141010 msgr2=0x7fe2480d2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 --2- 172.21.15.52:0/2950291400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe248141010 0x7fe2480d2000 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe2480b3820 tx=0x7fe23801c3c0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 -- 172.21.15.52:0/2950291400 shutdown_connections 2024-05-04T12:18:16.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 --2- 172.21.15.52:0/2950291400 >> v2:172.21.15.111:3300/0 conn(0x7fe248144570 0x7fe248143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 --2- 172.21.15.52:0/2950291400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe248141010 0x7fe2480d2000 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 -- 172.21.15.52:0/2950291400 >> 172.21.15.52:0/2950291400 conn(0x7fe248072be0 msgr2=0x7fe2480d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 -- 172.21.15.52:0/2950291400 shutdown_connections 2024-05-04T12:18:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 -- 172.21.15.52:0/2950291400 wait complete. 2024-05-04T12:18:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 Processor -- start 2024-05-04T12:18:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 -- start start 2024-05-04T12:18:16.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe2480d2540 0x7fe248153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe248141010 0x7fe248153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe248144570 0x7fe248153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe248141600 con 0x7fe248141010 2024-05-04T12:18:16.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe248142ad0 con 0x7fe248144570 2024-05-04T12:18:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24f18b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe24816b150 con 0x7fe2480d2540 2024-05-04T12:18:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe247fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe248141010 0x7fe248153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe247fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe248141010 0x7fe248153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54064/0 (socket says 172.21.15.52:54064) 2024-05-04T12:18:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe247fff640 1 -- 172.21.15.52:0/1287243987 learned_addr learned my addr 172.21.15.52:0/1287243987 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24d728640 1 --2- 172.21.15.52:0/1287243987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe248144570 0x7fe248153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe24cf27640 1 --2- 172.21.15.52:0/1287243987 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe2480d2540 0x7fe248153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe247fff640 1 -- 172.21.15.52:0/1287243987 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe2480d2540 msgr2=0x7fe248153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe247fff640 1 --2- 172.21.15.52:0/1287243987 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe2480d2540 0x7fe248153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe247fff640 1 -- 172.21.15.52:0/1287243987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe248144570 msgr2=0x7fe248153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe247fff640 1 --2- 172.21.15.52:0/1287243987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe248144570 0x7fe248153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe247fff640 1 -- 172.21.15.52:0/1287243987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24813ece0 con 0x7fe248141010 2024-05-04T12:18:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe247fff640 1 --2- 172.21.15.52:0/1287243987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe248141010 0x7fe248153990 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00b3f0 tx=0x7fe23c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe2457fa640 1 -- 172.21.15.52:0/1287243987 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c015020 con 0x7fe248141010 2024-05-04T12:18:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe2457fa640 1 -- 172.21.15.52:0/1287243987 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe23c011830 con 0x7fe248141010 2024-05-04T12:18:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.438+0000 7fe2457fa640 1 -- 172.21.15.52:0/1287243987 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c011c20 con 0x7fe248141010 2024-05-04T12:18:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.442+0000 7fe24f18b640 1 -- 172.21.15.52:0/1287243987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24813efc0 con 0x7fe248141010 2024-05-04T12:18:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.442+0000 7fe24f18b640 1 -- 172.21.15.52:0/1287243987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe24815bb70 con 0x7fe248141010 2024-05-04T12:18:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.442+0000 7fe24f18b640 1 -- 172.21.15.52:0/1287243987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe248062e50 con 0x7fe248141010 2024-05-04T12:18:16.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.442+0000 7fe2457fa640 1 -- 172.21.15.52:0/1287243987 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe23c018020 con 0x7fe248141010 2024-05-04T12:18:16.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.442+0000 7fe2457fa640 1 --2- 172.21.15.52:0/1287243987 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe2340678c0 0x7fe234069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.442+0000 7fe2457fa640 1 -- 172.21.15.52:0/1287243987 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fe23c085860 con 0x7fe248141010 2024-05-04T12:18:16.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.442+0000 7fe24cf27640 1 --2- 172.21.15.52:0/1287243987 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe2340678c0 0x7fe234069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.442+0000 7fe24cf27640 1 --2- 172.21.15.52:0/1287243987 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe2340678c0 0x7fe234069d80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2380ad000 tx=0x7fe2380a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:16.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.446+0000 7fe2457fa640 1 -- 172.21.15.52:0/1287243987 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe23c01f020 con 0x7fe248141010 2024-05-04T12:18:16.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.602+0000 7fe24f18b640 1 -- 172.21.15.52:0/1287243987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe24813f190 con 0x7fe248141010 2024-05-04T12:18:16.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.606+0000 7fe2457fa640 1 -- 172.21.15.52:0/1287243987 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7fe23c0566f0 con 0x7fe248141010 2024-05-04T12:18:16.606 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:18:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.606+0000 7fe24f18b640 1 -- 172.21.15.52:0/1287243987 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe2340678c0 msgr2=0x7fe234069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.606+0000 7fe24f18b640 1 --2- 172.21.15.52:0/1287243987 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe2340678c0 0x7fe234069d80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2380ad000 tx=0x7fe2380a6000 comp rx=0 tx=0).stop 2024-05-04T12:18:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.606+0000 7fe24f18b640 1 -- 172.21.15.52:0/1287243987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe248141010 msgr2=0x7fe248153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.606+0000 7fe24f18b640 1 --2- 172.21.15.52:0/1287243987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe248141010 0x7fe248153990 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00b3f0 tx=0x7fe23c00b8b0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.606+0000 7fe24f18b640 1 -- 172.21.15.52:0/1287243987 shutdown_connections 2024-05-04T12:18:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.606+0000 7fe24f18b640 1 --2- 172.21.15.52:0/1287243987 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe2340678c0 0x7fe234069d80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.606+0000 7fe24f18b640 1 --2- 172.21.15.52:0/1287243987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe248144570 0x7fe248153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.606+0000 7fe24f18b640 1 --2- 172.21.15.52:0/1287243987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe248141010 0x7fe248153990 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.606+0000 7fe24f18b640 1 --2- 172.21.15.52:0/1287243987 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe2480d2540 0x7fe248153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.606+0000 7fe24f18b640 1 -- 172.21.15.52:0/1287243987 >> 172.21.15.52:0/1287243987 conn(0x7fe248072be0 msgr2=0x7fe2481611b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:16.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.606+0000 7fe24f18b640 1 -- 172.21.15.52:0/1287243987 shutdown_connections 2024-05-04T12:18:16.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.606+0000 7fe24f18b640 1 -- 172.21.15.52:0/1287243987 wait complete. 2024-05-04T12:18:16.624 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T12:18:16.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.746+0000 7fbdc894f640 1 Processor -- start 2024-05-04T12:18:16.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.746+0000 7fbdc894f640 1 -- start start 2024-05-04T12:18:16.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.746+0000 7fbdc894f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbdc415e330 0x7fbdc40ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.746+0000 7fbdc894f640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdc415e7c0 con 0x7fbdc40cc2c0 2024-05-04T12:18:16.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.746+0000 7fbdc894f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdc415ea30 con 0x7fbdc4071be0 2024-05-04T12:18:16.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.746+0000 7fbdc894f640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fbdc406f080 con 0x7fbdc415e330 2024-05-04T12:18:16.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.746+0000 7fbdc259c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fbdc40cc2c0 0x7fbdc415a530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58852/0 (socket says 172.21.15.52:58852) 2024-05-04T12:18:16.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.746+0000 7fbdc259c640 1 -- 172.21.15.52:0/3668223533 learned_addr learned my addr 172.21.15.52:0/3668223533 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:16.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.746+0000 7fbdc1d9b640 1 --2- 172.21.15.52:0/3668223533 >> v2:172.21.15.52:3300/0 conn(0x7fbdc415e330 0x7fbdc40ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.746+0000 7fbdc159a640 1 -- 172.21.15.52:0/3668223533 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 962775460 0 0) 0x7fbdc415e7c0 con 0x7fbdc40cc2c0 2024-05-04T12:18:16.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.746+0000 7fbdc159a640 1 -- 172.21.15.52:0/3668223533 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdac003530 con 0x7fbdc40cc2c0 2024-05-04T12:18:16.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.746+0000 7fbdc1d9b640 1 -- 172.21.15.52:0/3668223533 >> v1:172.21.15.111:6789/0 conn(0x7fbdc4071be0 legacy=0x7fbdc41654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.746+0000 7fbdc1d9b640 1 -- 172.21.15.52:0/3668223533 >> v1:172.21.15.52:6789/0 conn(0x7fbdc40cc2c0 legacy=0x7fbdc415a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.746+0000 7fbdc1d9b640 1 -- 172.21.15.52:0/3668223533 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc415ed10 con 0x7fbdc415e330 2024-05-04T12:18:16.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc1d9b640 1 --2- 172.21.15.52:0/3668223533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc415e330 0x7fbdc40ccbd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80097c0 tx=0x7fbdb801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a523fe10276c5735 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:16.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc159a640 1 -- 172.21.15.52:0/3668223533 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80b0070 con 0x7fbdc415e330 2024-05-04T12:18:16.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc159a640 1 -- 172.21.15.52:0/3668223533 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdb80ab600 con 0x7fbdc415e330 2024-05-04T12:18:16.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc159a640 1 -- 172.21.15.52:0/3668223533 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80ab9d0 con 0x7fbdc415e330 2024-05-04T12:18:16.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 -- 172.21.15.52:0/3668223533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc415e330 msgr2=0x7fbdc40ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 --2- 172.21.15.52:0/3668223533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc415e330 0x7fbdc40ccbd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80097c0 tx=0x7fbdb801c4a0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 -- 172.21.15.52:0/3668223533 shutdown_connections 2024-05-04T12:18:16.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 --2- 172.21.15.52:0/3668223533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc415e330 0x7fbdc40ccbd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 -- 172.21.15.52:0/3668223533 >> 172.21.15.52:0/3668223533 conn(0x7fbdc40b14a0 msgr2=0x7fbdc40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:16.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 -- 172.21.15.52:0/3668223533 shutdown_connections 2024-05-04T12:18:16.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 -- 172.21.15.52:0/3668223533 wait complete. 2024-05-04T12:18:16.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 Processor -- start 2024-05-04T12:18:16.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 -- start start 2024-05-04T12:18:16.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc4071be0 0x7fbdc40aa420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdc40cc2c0 0x7fbdc41517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdc415e330 0x7fbdc414d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc40cd110 con 0x7fbdc4071be0 2024-05-04T12:18:16.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc406e740 con 0x7fbdc40cc2c0 2024-05-04T12:18:16.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc894f640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc40cd290 con 0x7fbdc415e330 2024-05-04T12:18:16.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc4071be0 0x7fbdc40aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc4071be0 0x7fbdc40aa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54074/0 (socket says 172.21.15.52:54074) 2024-05-04T12:18:16.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.750+0000 7fbdc259c640 1 -- 172.21.15.52:0/481486356 learned_addr learned my addr 172.21.15.52:0/481486356 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdc1d9b640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdc40cc2c0 0x7fbdc41517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdc2d9d640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdc415e330 0x7fbdc414d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdc259c640 1 -- 172.21.15.52:0/481486356 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdc415e330 msgr2=0x7fbdc414d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdc259c640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdc415e330 0x7fbdc414d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdc259c640 1 -- 172.21.15.52:0/481486356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdc40cc2c0 msgr2=0x7fbdc41517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdc259c640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdc40cc2c0 0x7fbdc41517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdc259c640 1 -- 172.21.15.52:0/481486356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc414e5d0 con 0x7fbdc4071be0 2024-05-04T12:18:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdc2d9d640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdc415e330 0x7fbdc414d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdc259c640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc4071be0 0x7fbdc40aa420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400c4d0 tx=0x7fbdb400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdb2ffd640 1 -- 172.21.15.52:0/481486356 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4017070 con 0x7fbdc4071be0 2024-05-04T12:18:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdb2ffd640 1 -- 172.21.15.52:0/481486356 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdb4005cf0 con 0x7fbdc4071be0 2024-05-04T12:18:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdc894f640 1 -- 172.21.15.52:0/481486356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc414e8b0 con 0x7fbdc4071be0 2024-05-04T12:18:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdb2ffd640 1 -- 172.21.15.52:0/481486356 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4007270 con 0x7fbdc4071be0 2024-05-04T12:18:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdc894f640 1 -- 172.21.15.52:0/481486356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc414ee70 con 0x7fbdc4071be0 2024-05-04T12:18:16.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdc894f640 1 -- 172.21.15.52:0/481486356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc4062e50 con 0x7fbdc4071be0 2024-05-04T12:18:16.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdb2ffd640 1 -- 172.21.15.52:0/481486356 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbdb401a070 con 0x7fbdc4071be0 2024-05-04T12:18:16.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdb2ffd640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbda40678c0 0x7fbda4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:16.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdb2ffd640 1 -- 172.21.15.52:0/481486356 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fbdb4084bb0 con 0x7fbdc4071be0 2024-05-04T12:18:16.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdc1d9b640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbda40678c0 0x7fbda4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:16.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.754+0000 7fbdc1d9b640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbda40678c0 0x7fbda4069d80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80ad000 tx=0x7fbdb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:16.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.758+0000 7fbdb2ffd640 1 -- 172.21.15.52:0/481486356 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbdb4055480 con 0x7fbdc4071be0 2024-05-04T12:18:16.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.958+0000 7fbdc894f640 1 -- 172.21.15.52:0/481486356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdc4075320 con 0x7fbdc4071be0 2024-05-04T12:18:16.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.958+0000 7fbdb2ffd640 1 -- 172.21.15.52:0/481486356 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7fbdb4059780 con 0x7fbdc4071be0 2024-05-04T12:18:16.961 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:16.961 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/163453709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6835","nonce":163453709},{"type":"v1","addr":"172.21.15.52:6837","nonce":163453709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-04T12:18:16.961 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 2 2024-05-04T12:18:16.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.962+0000 7fbdc894f640 1 -- 172.21.15.52:0/481486356 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbda40678c0 msgr2=0x7fbda4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.962+0000 7fbdc894f640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbda40678c0 0x7fbda4069d80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80ad000 tx=0x7fbdb8008040 comp rx=0 tx=0).stop 2024-05-04T12:18:16.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.962+0000 7fbdc894f640 1 -- 172.21.15.52:0/481486356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc4071be0 msgr2=0x7fbdc40aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:16.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.962+0000 7fbdc894f640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc4071be0 0x7fbdc40aa420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400c4d0 tx=0x7fbdb400c990 comp rx=0 tx=0).stop 2024-05-04T12:18:16.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.962+0000 7fbdc894f640 1 -- 172.21.15.52:0/481486356 shutdown_connections 2024-05-04T12:18:16.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.962+0000 7fbdc894f640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbda40678c0 0x7fbda4069d80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.962+0000 7fbdc894f640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdc415e330 0x7fbdc414d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.962+0000 7fbdc894f640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdc40cc2c0 0x7fbdc41517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.962+0000 7fbdc894f640 1 --2- 172.21.15.52:0/481486356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc4071be0 0x7fbdc40aa420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:16.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.962+0000 7fbdc894f640 1 -- 172.21.15.52:0/481486356 >> 172.21.15.52:0/481486356 conn(0x7fbdc40b14a0 msgr2=0x7fbdc40b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:16.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.962+0000 7fbdc894f640 1 -- 172.21.15.52:0/481486356 shutdown_connections 2024-05-04T12:18:16.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:16.962+0000 7fbdc894f640 1 -- 172.21.15.52:0/481486356 wait complete. 2024-05-04T12:18:18.980 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T12:18:19.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff8524ed640 1 Processor -- start 2024-05-04T12:18:19.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff8524ed640 1 -- start start 2024-05-04T12:18:19.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff8524ed640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff84c067070 0x7ff84c0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff8524ed640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff84c0ceff0 0x7ff84c15e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff8524ed640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff84c0cb650 con 0x7ff84c0cec10 2024-05-04T12:18:19.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff8524ed640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7ff84c0cb930 con 0x7ff84c0ceff0 2024-05-04T12:18:19.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff8524ed640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7ff84c0cbc10 con 0x7ff84c067070 2024-05-04T12:18:19.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff84b7fe640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff84c067070 0x7ff84c0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff850a8a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff84c0ceff0 0x7ff84c15e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff84b7fe640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff84c067070 0x7ff84c0ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.52:51428/0 (socket says 172.21.15.52:51428) 2024-05-04T12:18:19.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff84bfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff84c0cec10 0x7ff84c0d0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.52:57610/0 (socket says 172.21.15.52:57610) 2024-05-04T12:18:19.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff850a8a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff84c0ceff0 0x7ff84c15e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:57638/0 (socket says 172.21.15.52:57638) 2024-05-04T12:18:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff84b7fe640 1 -- 172.21.15.52:0/1073319457 learned_addr learned my addr 172.21.15.52:0/1073319457 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff850a8a640 1 -- 172.21.15.52:0/1073319457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff84c067070 msgr2=0x7ff84c0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff850a8a640 1 --2- 172.21.15.52:0/1073319457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff84c067070 0x7ff84c0ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff850a8a640 1 -- 172.21.15.52:0/1073319457 >> v1:172.21.15.111:6789/0 conn(0x7ff84c0cec10 legacy=0x7ff84c0d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff84affd640 1 -- 172.21.15.52:0/1073319457 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3146994817 0 0) 0x7ff84c0cb650 con 0x7ff84c0cec10 2024-05-04T12:18:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.114+0000 7ff850a8a640 1 -- 172.21.15.52:0/1073319457 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c0b0570 con 0x7ff84c0ceff0 2024-05-04T12:18:19.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.118+0000 7ff850a8a640 1 --2- 172.21.15.52:0/1073319457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff84c0ceff0 0x7ff84c15e060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff844009d90 tx=0x7ff84401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e254244a7ae85754 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:19.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.118+0000 7ff84affd640 1 -- 172.21.15.52:0/1073319457 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8440b2070 con 0x7ff84c0ceff0 2024-05-04T12:18:19.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.118+0000 7ff84affd640 1 -- 172.21.15.52:0/1073319457 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8440a5940 con 0x7ff84c0ceff0 2024-05-04T12:18:19.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.118+0000 7ff84affd640 1 -- 172.21.15.52:0/1073319457 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8440a5d10 con 0x7ff84c0ceff0 2024-05-04T12:18:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.118+0000 7ff8524ed640 1 -- 172.21.15.52:0/1073319457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff84c0ceff0 msgr2=0x7ff84c15e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.118+0000 7ff8524ed640 1 --2- 172.21.15.52:0/1073319457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff84c0ceff0 0x7ff84c15e060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff844009d90 tx=0x7ff84401c970 comp rx=0 tx=0).stop 2024-05-04T12:18:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.118+0000 7ff8524ed640 1 -- 172.21.15.52:0/1073319457 shutdown_connections 2024-05-04T12:18:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.118+0000 7ff8524ed640 1 --2- 172.21.15.52:0/1073319457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff84c0ceff0 0x7ff84c15e060 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.118+0000 7ff8524ed640 1 --2- 172.21.15.52:0/1073319457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff84c067070 0x7ff84c0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.118+0000 7ff8524ed640 1 -- 172.21.15.52:0/1073319457 >> 172.21.15.52:0/1073319457 conn(0x7ff84c0cd5b0 msgr2=0x7ff84c0cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.118+0000 7ff8524ed640 1 -- 172.21.15.52:0/1073319457 shutdown_connections 2024-05-04T12:18:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.118+0000 7ff8524ed640 1 -- 172.21.15.52:0/1073319457 wait complete. 2024-05-04T12:18:19.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.118+0000 7ff8524ed640 1 Processor -- start 2024-05-04T12:18:19.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff8524ed640 1 -- start start 2024-05-04T12:18:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff8524ed640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff84c067070 0x7ff84c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff8524ed640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff84c0cec10 0x7ff84c14d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff8524ed640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff84c0ceff0 0x7ff84c15c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff8524ed640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c0b1630 con 0x7ff84c067070 2024-05-04T12:18:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff8524ed640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c0b14b0 con 0x7ff84c0ceff0 2024-05-04T12:18:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff8524ed640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7ff84c0b17b0 con 0x7ff84c0cec10 2024-05-04T12:18:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff84bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff84c067070 0x7ff84c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff84bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff84c067070 0x7ff84c070600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54088/0 (socket says 172.21.15.52:54088) 2024-05-04T12:18:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff84bfff640 1 -- 172.21.15.52:0/3320457689 learned_addr learned my addr 172.21.15.52:0/3320457689 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff850a8a640 1 --2- 172.21.15.52:0/3320457689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff84c0ceff0 0x7ff84c15c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff84b7fe640 1 --2- 172.21.15.52:0/3320457689 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff84c0cec10 0x7ff84c14d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff84bfff640 1 -- 172.21.15.52:0/3320457689 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff84c0cec10 msgr2=0x7ff84c14d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff84bfff640 1 --2- 172.21.15.52:0/3320457689 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff84c0cec10 0x7ff84c14d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff84bfff640 1 -- 172.21.15.52:0/3320457689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff84c0ceff0 msgr2=0x7ff84c15c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff84bfff640 1 --2- 172.21.15.52:0/3320457689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff84c0ceff0 0x7ff84c15c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff84bfff640 1 -- 172.21.15.52:0/3320457689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c071e10 con 0x7ff84c067070 2024-05-04T12:18:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff84bfff640 1 --2- 172.21.15.52:0/3320457689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff84c067070 0x7ff84c070600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00c4b0 tx=0x7ff83c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff848ff9640 1 -- 172.21.15.52:0/3320457689 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c018070 con 0x7ff84c067070 2024-05-04T12:18:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff848ff9640 1 -- 172.21.15.52:0/3320457689 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff83c007500 con 0x7ff84c067070 2024-05-04T12:18:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff8524ed640 1 -- 172.21.15.52:0/3320457689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84c0c6b00 con 0x7ff84c067070 2024-05-04T12:18:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff848ff9640 1 -- 172.21.15.52:0/3320457689 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c004a30 con 0x7ff84c067070 2024-05-04T12:18:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff8524ed640 1 -- 172.21.15.52:0/3320457689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84c072390 con 0x7ff84c067070 2024-05-04T12:18:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff8524ed640 1 -- 172.21.15.52:0/3320457689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84c062e50 con 0x7ff84c067070 2024-05-04T12:18:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff848ff9640 1 -- 172.21.15.52:0/3320457689 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff83c004bd0 con 0x7ff84c067070 2024-05-04T12:18:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff848ff9640 1 --2- 172.21.15.52:0/3320457689 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff8380678f0 0x7ff838069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff848ff9640 1 -- 172.21.15.52:0/3320457689 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff83c084b40 con 0x7ff84c067070 2024-05-04T12:18:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.122+0000 7ff84b7fe640 1 --2- 172.21.15.52:0/3320457689 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff8380678f0 0x7ff838069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.126+0000 7ff84b7fe640 1 --2- 172.21.15.52:0/3320457689 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff8380678f0 0x7ff838069db0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff84c14f5c0 tx=0x7ff840008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.130+0000 7ff848ff9640 1 -- 172.21.15.52:0/3320457689 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff83c055950 con 0x7ff84c067070 2024-05-04T12:18:19.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.322+0000 7ff8524ed640 1 -- 172.21.15.52:0/3320457689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff84c075320 con 0x7ff84c067070 2024-05-04T12:18:19.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.322+0000 7ff848ff9640 1 -- 172.21.15.52:0/3320457689 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3864 (secure 0 0 0) 0x7ff83c055250 con 0x7ff84c067070 2024-05-04T12:18:19.325 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:19.325 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6836/482177111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":482177111},{"type":"v1","addr":"172.21.15.52:6836","nonce":482177111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/163453709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6835","nonce":163453709},{"type":"v1","addr":"172.21.15.52:6837","nonce":163453709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3596668218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3596668218},{"type":"v1","addr":"172.21.15.111:6833","nonce":3596668218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1656435333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1656435333},{"type":"v1","addr":"172.21.15.111:6835","nonce":1656435333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-04T12:18:19.326 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 3 2024-05-04T12:18:19.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.326+0000 7ff8524ed640 1 -- 172.21.15.52:0/3320457689 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff8380678f0 msgr2=0x7ff838069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.326+0000 7ff8524ed640 1 --2- 172.21.15.52:0/3320457689 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff8380678f0 0x7ff838069db0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff84c14f5c0 tx=0x7ff840008040 comp rx=0 tx=0).stop 2024-05-04T12:18:19.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.326+0000 7ff8524ed640 1 -- 172.21.15.52:0/3320457689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff84c067070 msgr2=0x7ff84c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.326+0000 7ff8524ed640 1 --2- 172.21.15.52:0/3320457689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff84c067070 0x7ff84c070600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00c4b0 tx=0x7ff83c00c970 comp rx=0 tx=0).stop 2024-05-04T12:18:19.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.326+0000 7ff8524ed640 1 -- 172.21.15.52:0/3320457689 shutdown_connections 2024-05-04T12:18:19.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.326+0000 7ff8524ed640 1 --2- 172.21.15.52:0/3320457689 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff8380678f0 0x7ff838069db0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.326+0000 7ff8524ed640 1 --2- 172.21.15.52:0/3320457689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff84c0ceff0 0x7ff84c15c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.326+0000 7ff8524ed640 1 --2- 172.21.15.52:0/3320457689 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff84c0cec10 0x7ff84c14d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.326+0000 7ff8524ed640 1 --2- 172.21.15.52:0/3320457689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff84c067070 0x7ff84c070600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.326+0000 7ff8524ed640 1 -- 172.21.15.52:0/3320457689 >> 172.21.15.52:0/3320457689 conn(0x7ff84c0cd5b0 msgr2=0x7ff84c159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:19.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.326+0000 7ff8524ed640 1 -- 172.21.15.52:0/3320457689 shutdown_connections 2024-05-04T12:18:19.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.326+0000 7ff8524ed640 1 -- 172.21.15.52:0/3320457689 wait complete. 2024-05-04T12:18:19.345 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:19.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b77f0640 1 Processor -- start 2024-05-04T12:18:19.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b77f0640 1 -- start start 2024-05-04T12:18:19.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b77f0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f91b015bf50 0x7f91b015c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b77f0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f91b00aa0c0 0x7f91b00ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b77f0640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91b006d1b0 con 0x7f91b00aa550 2024-05-04T12:18:19.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b77f0640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f91b006d490 con 0x7f91b015bf50 2024-05-04T12:18:19.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b77f0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f91b006d770 con 0x7f91b00aa0c0 2024-05-04T12:18:19.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b558c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f91b015bf50 0x7f91b015c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b5d8d640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f91b00aa550 0x7f91b006e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58860/0 (socket says 172.21.15.52:58860) 2024-05-04T12:18:19.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b5d8d640 1 -- 172.21.15.52:0/1604542 learned_addr learned my addr 172.21.15.52:0/1604542 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:19.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b4d8b640 1 --2- 172.21.15.52:0/1604542 >> v2:172.21.15.111:3300/0 conn(0x7f91b00aa0c0 0x7f91b00ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b558c640 1 -- 172.21.15.52:0/1604542 >> v2:172.21.15.111:3300/0 conn(0x7f91b00aa0c0 msgr2=0x7f91b00ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:19.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b558c640 1 --2- 172.21.15.52:0/1604542 >> v2:172.21.15.111:3300/0 conn(0x7f91b00aa0c0 0x7f91b00ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b558c640 1 -- 172.21.15.52:0/1604542 >> v1:172.21.15.52:6789/0 conn(0x7f91b00aa550 legacy=0x7f91b006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91a7fff640 1 -- 172.21.15.52:0/1604542 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2813759934 0 0) 0x7f91b006d1b0 con 0x7f91b00aa550 2024-05-04T12:18:19.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b558c640 1 -- 172.21.15.52:0/1604542 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b006da50 con 0x7f91b015bf50 2024-05-04T12:18:19.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.470+0000 7f91b4d8b640 1 --2- 172.21.15.52:0/1604542 >> v2:172.21.15.111:3300/0 conn(0x7f91b00aa0c0 0x7f91b00ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:19.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.474+0000 7f91b558c640 1 --2- 172.21.15.52:0/1604542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b015bf50 0x7f91b015c330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f91b00b3ae0 tx=0x7f91ac01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194c34fbcf24a072 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:19.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.474+0000 7f91a7fff640 1 -- 172.21.15.52:0/1604542 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac0a7070 con 0x7f91b015bf50 2024-05-04T12:18:19.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.474+0000 7f91a7fff640 1 -- 172.21.15.52:0/1604542 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91ac0b0070 con 0x7f91b015bf50 2024-05-04T12:18:19.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.474+0000 7f91a7fff640 1 -- 172.21.15.52:0/1604542 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac0ab890 con 0x7f91b015bf50 2024-05-04T12:18:19.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.474+0000 7f91b77f0640 1 -- 172.21.15.52:0/1604542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b015bf50 msgr2=0x7f91b015c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.474+0000 7f91b77f0640 1 --2- 172.21.15.52:0/1604542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b015bf50 0x7f91b015c330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f91b00b3ae0 tx=0x7f91ac01c450 comp rx=0 tx=0).stop 2024-05-04T12:18:19.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.474+0000 7f91b77f0640 1 -- 172.21.15.52:0/1604542 shutdown_connections 2024-05-04T12:18:19.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.474+0000 7f91b77f0640 1 --2- 172.21.15.52:0/1604542 >> v2:172.21.15.111:3300/0 conn(0x7f91b00aa0c0 0x7f91b00ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.474+0000 7f91b77f0640 1 --2- 172.21.15.52:0/1604542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b015bf50 0x7f91b015c330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.474+0000 7f91b77f0640 1 -- 172.21.15.52:0/1604542 >> 172.21.15.52:0/1604542 conn(0x7f91b00b14a0 msgr2=0x7f91b00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:19.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.474+0000 7f91b77f0640 1 -- 172.21.15.52:0/1604542 shutdown_connections 2024-05-04T12:18:19.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.474+0000 7f91b77f0640 1 -- 172.21.15.52:0/1604542 wait complete. 2024-05-04T12:18:19.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.474+0000 7f91b77f0640 1 Processor -- start 2024-05-04T12:18:19.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.474+0000 7f91b77f0640 1 -- start start 2024-05-04T12:18:19.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b77f0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b00aa0c0 0x7f91b0145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b77f0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b00aa550 0x7f91b0140a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b77f0640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91b015bf50 0x7f91b014e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b77f0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f91b0169030 con 0x7f91b00aa0c0 2024-05-04T12:18:19.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b77f0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f91b00cd4a0 con 0x7f91b00aa550 2024-05-04T12:18:19.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b77f0640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f91b01691b0 con 0x7f91b015bf50 2024-05-04T12:18:19.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b558c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b00aa0c0 0x7f91b0145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b558c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b00aa0c0 0x7f91b0145a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54104/0 (socket says 172.21.15.52:54104) 2024-05-04T12:18:19.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b558c640 1 -- 172.21.15.52:0/4120660866 learned_addr learned my addr 172.21.15.52:0/4120660866 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:19.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b4d8b640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b00aa550 0x7f91b0140a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b5d8d640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91b015bf50 0x7f91b014e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b558c640 1 -- 172.21.15.52:0/4120660866 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91b015bf50 msgr2=0x7f91b014e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b558c640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91b015bf50 0x7f91b014e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b558c640 1 -- 172.21.15.52:0/4120660866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b00aa550 msgr2=0x7f91b0140a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b558c640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b00aa550 0x7f91b0140a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b558c640 1 -- 172.21.15.52:0/4120660866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b0140f70 con 0x7f91b00aa0c0 2024-05-04T12:18:19.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b4d8b640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b00aa550 0x7f91b0140a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:19.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b5d8d640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91b015bf50 0x7f91b014e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:19.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b558c640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b00aa0c0 0x7f91b0145a90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f91ac019060 tx=0x7f91ac01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:19.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91a5ffb640 1 -- 172.21.15.52:0/4120660866 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac006e20 con 0x7f91b00aa0c0 2024-05-04T12:18:19.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91a5ffb640 1 -- 172.21.15.52:0/4120660866 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91ac004220 con 0x7f91b00aa0c0 2024-05-04T12:18:19.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b77f0640 1 -- 172.21.15.52:0/4120660866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b0141220 con 0x7f91b00aa0c0 2024-05-04T12:18:19.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91a5ffb640 1 -- 172.21.15.52:0/4120660866 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac003620 con 0x7f91b00aa0c0 2024-05-04T12:18:19.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b77f0640 1 -- 172.21.15.52:0/4120660866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b01417c0 con 0x7f91b00aa0c0 2024-05-04T12:18:19.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b77f0640 1 -- 172.21.15.52:0/4120660866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b0062e50 con 0x7f91b00aa0c0 2024-05-04T12:18:19.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91a5ffb640 1 -- 172.21.15.52:0/4120660866 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f91ac0037c0 con 0x7f91b00aa0c0 2024-05-04T12:18:19.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91a5ffb640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f91940678c0 0x7f9194069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91a5ffb640 1 -- 172.21.15.52:0/4120660866 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f91ac1200a0 con 0x7f91b00aa0c0 2024-05-04T12:18:19.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.478+0000 7f91b4d8b640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f91940678c0 0x7f9194069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.482+0000 7f91b4d8b640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f91940678c0 0x7f9194069d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f91b0151e70 tx=0x7f91a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:19.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.482+0000 7f91a5ffb640 1 -- 172.21.15.52:0/4120660866 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f91ac016070 con 0x7f91b00aa0c0 2024-05-04T12:18:19.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.646+0000 7f91b77f0640 1 -- 172.21.15.52:0/4120660866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91b0072b60 con 0x7f91b00aa0c0 2024-05-04T12:18:19.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.646+0000 7f91a5ffb640 1 -- 172.21.15.52:0/4120660866 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7f91ac0f10e0 con 0x7f91b00aa0c0 2024-05-04T12:18:19.650 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:19.650 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:13.269413+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.650+0000 7f91b77f0640 1 -- 172.21.15.52:0/4120660866 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f91940678c0 msgr2=0x7f9194069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.650+0000 7f91b77f0640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f91940678c0 0x7f9194069d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f91b0151e70 tx=0x7f91a0008040 comp rx=0 tx=0).stop 2024-05-04T12:18:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.650+0000 7f91b77f0640 1 -- 172.21.15.52:0/4120660866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b00aa0c0 msgr2=0x7f91b0145a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.650+0000 7f91b77f0640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b00aa0c0 0x7f91b0145a90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f91ac019060 tx=0x7f91ac01ac90 comp rx=0 tx=0).stop 2024-05-04T12:18:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.650+0000 7f91b77f0640 1 -- 172.21.15.52:0/4120660866 shutdown_connections 2024-05-04T12:18:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.650+0000 7f91b77f0640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f91940678c0 0x7f9194069d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.650+0000 7f91b77f0640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91b015bf50 0x7f91b014e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.650+0000 7f91b77f0640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b00aa550 0x7f91b0140a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.650+0000 7f91b77f0640 1 --2- 172.21.15.52:0/4120660866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b00aa0c0 0x7f91b0145a90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.650+0000 7f91b77f0640 1 -- 172.21.15.52:0/4120660866 >> 172.21.15.52:0/4120660866 conn(0x7f91b00b14a0 msgr2=0x7f91b0142200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.650+0000 7f91b77f0640 1 -- 172.21.15.52:0/4120660866 shutdown_connections 2024-05-04T12:18:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.650+0000 7f91b77f0640 1 -- 172.21.15.52:0/4120660866 wait complete. 2024-05-04T12:18:19.667 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:18:19.667 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:18:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.782+0000 7f1b463b1640 1 Processor -- start 2024-05-04T12:18:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.782+0000 7f1b463b1640 1 -- start start 2024-05-04T12:18:19.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.782+0000 7f1b463b1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1b4015c590 0x7f1b400cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.782+0000 7f1b463b1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b4006dbf0 con 0x7f1b400cccd0 2024-05-04T12:18:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.782+0000 7f1b463b1640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b4006cfe0 con 0x7f1b40145960 2024-05-04T12:18:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.782+0000 7f1b463b1640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f1b400cd140 con 0x7f1b4015c590 2024-05-04T12:18:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.782+0000 7f1b3f7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1b4015c590 0x7f1b400cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.782+0000 7f1b4494e640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f1b400cccd0 0x7f1b4006e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58866/0 (socket says 172.21.15.52:58866) 2024-05-04T12:18:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.782+0000 7f1b4494e640 1 -- 172.21.15.52:0/1589310576 learned_addr learned my addr 172.21.15.52:0/1589310576 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.782+0000 7f1b3effd640 1 -- 172.21.15.52:0/1589310576 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 78072695 0 0) 0x7f1b4006dbf0 con 0x7f1b400cccd0 2024-05-04T12:18:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.782+0000 7f1b3effd640 1 -- 172.21.15.52:0/1589310576 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b28003530 con 0x7f1b400cccd0 2024-05-04T12:18:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.782+0000 7f1b3f7fe640 1 -- 172.21.15.52:0/1589310576 >> v1:172.21.15.111:6789/0 conn(0x7f1b40145960 legacy=0x7f1b40160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.782+0000 7f1b3f7fe640 1 -- 172.21.15.52:0/1589310576 >> v1:172.21.15.52:6789/0 conn(0x7f1b400cccd0 legacy=0x7f1b4006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.782+0000 7f1b3f7fe640 1 -- 172.21.15.52:0/1589310576 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4006d2c0 con 0x7f1b4015c590 2024-05-04T12:18:19.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b3f7fe640 1 --2- 172.21.15.52:0/1589310576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b4015c590 0x7f1b400cc790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009d90 tx=0x7f1b3401c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c412afe43d76a2c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:19.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b3effd640 1 -- 172.21.15.52:0/1589310576 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b340a66c0 con 0x7f1b4015c590 2024-05-04T12:18:19.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b3effd640 1 -- 172.21.15.52:0/1589310576 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b340a6860 con 0x7f1b4015c590 2024-05-04T12:18:19.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b3effd640 1 -- 172.21.15.52:0/1589310576 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b340a6c30 con 0x7f1b4015c590 2024-05-04T12:18:19.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 -- 172.21.15.52:0/1589310576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b4015c590 msgr2=0x7f1b400cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 --2- 172.21.15.52:0/1589310576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b4015c590 0x7f1b400cc790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009d90 tx=0x7f1b3401c5f0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 -- 172.21.15.52:0/1589310576 shutdown_connections 2024-05-04T12:18:19.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 --2- 172.21.15.52:0/1589310576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b4015c590 0x7f1b400cc790 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 -- 172.21.15.52:0/1589310576 >> 172.21.15.52:0/1589310576 conn(0x7f1b40067730 msgr2=0x7f1b400b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:19.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 -- 172.21.15.52:0/1589310576 shutdown_connections 2024-05-04T12:18:19.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 -- 172.21.15.52:0/1589310576 wait complete. 2024-05-04T12:18:19.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 Processor -- start 2024-05-04T12:18:19.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 -- start start 2024-05-04T12:18:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b400cccd0 0x7f1b40141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b40145960 0x7f1b40158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b4015c590 0x7f1b4013b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4006cdc0 con 0x7f1b4015c590 2024-05-04T12:18:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4006cc40 con 0x7f1b400cccd0 2024-05-04T12:18:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b463b1640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f1b40145d40 con 0x7f1b40145960 2024-05-04T12:18:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b4494e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b4015c590 0x7f1b4013b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b4494e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b4015c590 0x7f1b4013b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54112/0 (socket says 172.21.15.52:54112) 2024-05-04T12:18:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b4494e640 1 -- 172.21.15.52:0/1148464168 learned_addr learned my addr 172.21.15.52:0/1148464168 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b3ffff640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b400cccd0 0x7f1b40141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b3f7fe640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b40145960 0x7f1b40158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b4494e640 1 -- 172.21.15.52:0/1148464168 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b40145960 msgr2=0x7f1b40158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b4494e640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b40145960 0x7f1b40158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b4494e640 1 -- 172.21.15.52:0/1148464168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b400cccd0 msgr2=0x7f1b40141670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b4494e640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b400cccd0 0x7f1b40141670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b4494e640 1 -- 172.21.15.52:0/1148464168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b401555a0 con 0x7f1b4015c590 2024-05-04T12:18:19.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b3ffff640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b400cccd0 0x7f1b40141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:19.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b4494e640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b4015c590 0x7f1b4013b0b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c3c0 tx=0x7f1b3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:19.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.786+0000 7f1b3cff9640 1 -- 172.21.15.52:0/1148464168 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38017070 con 0x7f1b4015c590 2024-05-04T12:18:19.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.790+0000 7f1b3cff9640 1 -- 172.21.15.52:0/1148464168 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b38005cf0 con 0x7f1b4015c590 2024-05-04T12:18:19.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.790+0000 7f1b463b1640 1 -- 172.21.15.52:0/1148464168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b40154430 con 0x7f1b4015c590 2024-05-04T12:18:19.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.790+0000 7f1b3cff9640 1 -- 172.21.15.52:0/1148464168 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38007310 con 0x7f1b4015c590 2024-05-04T12:18:19.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.790+0000 7f1b463b1640 1 -- 172.21.15.52:0/1148464168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b40141bb0 con 0x7f1b4015c590 2024-05-04T12:18:19.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.790+0000 7f1b463b1640 1 -- 172.21.15.52:0/1148464168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b40062e50 con 0x7f1b4015c590 2024-05-04T12:18:19.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.790+0000 7f1b3cff9640 1 -- 172.21.15.52:0/1148464168 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1b380074b0 con 0x7f1b4015c590 2024-05-04T12:18:19.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.790+0000 7f1b3cff9640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1b2c0678c0 0x7f1b2c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:19.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.790+0000 7f1b3ffff640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1b2c0678c0 0x7f1b2c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:19.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.790+0000 7f1b3ffff640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1b2c0678c0 0x7f1b2c069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1b300016f0 tx=0x7f1b30009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:19.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.790+0000 7f1b3cff9640 1 -- 172.21.15.52:0/1148464168 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f1b3805c170 con 0x7f1b4015c590 2024-05-04T12:18:19.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.798+0000 7f1b3cff9640 1 -- 172.21.15.52:0/1148464168 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1b38020020 con 0x7f1b4015c590 2024-05-04T12:18:19.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.954+0000 7f1b463b1640 1 -- 172.21.15.52:0/1148464168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1b400cc270 con 0x7f1b4015c590 2024-05-04T12:18:19.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.954+0000 7f1b3cff9640 1 -- 172.21.15.52:0/1148464168 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f1b38055fd0 con 0x7f1b4015c590 2024-05-04T12:18:19.956 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:18:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.954+0000 7f1b463b1640 1 -- 172.21.15.52:0/1148464168 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1b2c0678c0 msgr2=0x7f1b2c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.954+0000 7f1b463b1640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1b2c0678c0 0x7f1b2c069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1b300016f0 tx=0x7f1b30009040 comp rx=0 tx=0).stop 2024-05-04T12:18:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.954+0000 7f1b463b1640 1 -- 172.21.15.52:0/1148464168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b4015c590 msgr2=0x7f1b4013b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.954+0000 7f1b463b1640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b4015c590 0x7f1b4013b0b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c3c0 tx=0x7f1b3800c880 comp rx=0 tx=0).stop 2024-05-04T12:18:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.954+0000 7f1b463b1640 1 -- 172.21.15.52:0/1148464168 shutdown_connections 2024-05-04T12:18:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.954+0000 7f1b463b1640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1b2c0678c0 0x7f1b2c069d80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.954+0000 7f1b463b1640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b4015c590 0x7f1b4013b0b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.954+0000 7f1b463b1640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b40145960 0x7f1b40158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.954+0000 7f1b463b1640 1 --2- 172.21.15.52:0/1148464168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b400cccd0 0x7f1b40141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:19.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.954+0000 7f1b463b1640 1 -- 172.21.15.52:0/1148464168 >> 172.21.15.52:0/1148464168 conn(0x7f1b40067730 msgr2=0x7f1b40144aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:19.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.954+0000 7f1b463b1640 1 -- 172.21.15.52:0/1148464168 shutdown_connections 2024-05-04T12:18:19.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:19.954+0000 7f1b463b1640 1 -- 172.21.15.52:0/1148464168 wait complete. 2024-05-04T12:18:19.973 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-04T12:18:20.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.094+0000 7f8f10d19640 1 Processor -- start 2024-05-04T12:18:20.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.094+0000 7f8f10d19640 1 -- start start 2024-05-04T12:18:20.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f10d19640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f0c0cd260 0x7f8f0c06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f10d19640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f0c06d1b0 con 0x7f8f0c15bf50 2024-05-04T12:18:20.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f10d19640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f0c06d490 con 0x7f8f0c0aa0c0 2024-05-04T12:18:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f10d19640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f8f0c15ca40 con 0x7f8f0c0cd260 2024-05-04T12:18:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0a59c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f8f0c15bf50 0x7f8f0c15c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58878/0 (socket says 172.21.15.52:58878) 2024-05-04T12:18:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0a59c640 1 -- 172.21.15.52:0/3512645938 learned_addr learned my addr 172.21.15.52:0/3512645938 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0ad9d640 1 --2- 172.21.15.52:0/3512645938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f0c0cd260 0x7f8f0c06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 -- 172.21.15.52:0/3512645938 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 727943921 0 0) 0x7f8f0c06d1b0 con 0x7f8f0c15bf50 2024-05-04T12:18:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 -- 172.21.15.52:0/3512645938 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ef8003530 con 0x7f8f0c15bf50 2024-05-04T12:18:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 -- 172.21.15.52:0/3512645938 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 630651005 0 0) 0x7f8ef8003530 con 0x7f8f0c15bf50 2024-05-04T12:18:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 -- 172.21.15.52:0/3512645938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f0c0cd260 msgr2=0x7f8f0c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 --2- 172.21.15.52:0/3512645938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f0c0cd260 0x7f8f0c06e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 -- 172.21.15.52:0/3512645938 >> v1:172.21.15.111:6789/0 conn(0x7f8f0c0aa0c0 legacy=0x7f8f0c0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 -- 172.21.15.52:0/3512645938 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0c06d770 con 0x7f8f0c15bf50 2024-05-04T12:18:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 -- 172.21.15.52:0/3512645938 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f8efc001680 con 0x7f8f0c15bf50 2024-05-04T12:18:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 -- 172.21.15.52:0/3512645938 >> v1:172.21.15.52:6789/0 conn(0x7f8f0c15bf50 legacy=0x7f8f0c15c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 --2- 172.21.15.52:0/3512645938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ef80039f0 0x7f8ef8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 --2- 172.21.15.52:0/3512645938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ef8004810 0x7f8ef8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 --2- 172.21.15.52:0/3512645938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f0c0cd260 0x7f8ef80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 -- 172.21.15.52:0/3512645938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c15ca40 con 0x7f8ef80039f0 2024-05-04T12:18:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 -- 172.21.15.52:0/3512645938 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef800bab0 con 0x7f8ef8004810 2024-05-04T12:18:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0959a640 1 -- 172.21.15.52:0/3512645938 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef800bcb0 con 0x7f8f0c0cd260 2024-05-04T12:18:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0a59c640 1 --2- 172.21.15.52:0/3512645938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ef80039f0 0x7f8ef8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f09d9b640 1 --2- 172.21.15.52:0/3512645938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ef8004810 0x7f8ef8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0ad9d640 1 --2- 172.21.15.52:0/3512645938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f0c0cd260 0x7f8ef80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0a59c640 1 -- 172.21.15.52:0/3512645938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f0c0cd260 msgr2=0x7f8ef80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0a59c640 1 --2- 172.21.15.52:0/3512645938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f0c0cd260 0x7f8ef80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0a59c640 1 -- 172.21.15.52:0/3512645938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ef8004810 msgr2=0x7f8ef8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:20.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0a59c640 1 --2- 172.21.15.52:0/3512645938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ef8004810 0x7f8ef8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0a59c640 1 -- 172.21.15.52:0/3512645938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef8003530 con 0x7f8ef80039f0 2024-05-04T12:18:20.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0ad9d640 1 --2- 172.21.15.52:0/3512645938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f0c0cd260 0x7f8ef80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:20.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.098+0000 7f8f0a59c640 1 --2- 172.21.15.52:0/3512645938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ef80039f0 0x7f8ef8003e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00c210 tx=0x7f8efc01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d40463f78877e92 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:20.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f0959a640 1 -- 172.21.15.52:0/3512645938 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc007d20 con 0x7f8ef80039f0 2024-05-04T12:18:20.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f0959a640 1 -- 172.21.15.52:0/3512645938 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8efc0a9070 con 0x7f8ef80039f0 2024-05-04T12:18:20.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f0959a640 1 -- 172.21.15.52:0/3512645938 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc005d20 con 0x7f8ef80039f0 2024-05-04T12:18:20.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 -- 172.21.15.52:0/3512645938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ef80039f0 msgr2=0x7f8ef8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 --2- 172.21.15.52:0/3512645938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ef80039f0 0x7f8ef8003e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00c210 tx=0x7f8efc01cc10 comp rx=0 tx=0).stop 2024-05-04T12:18:20.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f0ad9d640 1 -- 172.21.15.52:0/3512645938 reap_dead start 2024-05-04T12:18:20.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 -- 172.21.15.52:0/3512645938 shutdown_connections 2024-05-04T12:18:20.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 -- 172.21.15.52:0/3512645938 >> 172.21.15.52:0/3512645938 conn(0x7f8f0c0b14a0 msgr2=0x7f8f0c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:20.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 -- 172.21.15.52:0/3512645938 shutdown_connections 2024-05-04T12:18:20.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 -- 172.21.15.52:0/3512645938 wait complete. 2024-05-04T12:18:20.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 Processor -- start 2024-05-04T12:18:20.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 -- start start 2024-05-04T12:18:20.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f0c143b60 0x7f8f0c071050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f0c156e60 0x7f8f0c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f0c144220 0x7f8f0c1446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c06ebe0 con 0x7f8f0c156e60 2024-05-04T12:18:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c06ea60 con 0x7f8f0c143b60 2024-05-04T12:18:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f8f0c06ed60 con 0x7f8f0c144220 2024-05-04T12:18:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f09d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f0c156e60 0x7f8f0c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f09d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f0c156e60 0x7f8f0c1572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54118/0 (socket says 172.21.15.52:54118) 2024-05-04T12:18:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f09d9b640 1 -- 172.21.15.52:0/3018255790 learned_addr learned my addr 172.21.15.52:0/3018255790 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f0ad9d640 1 --2- 172.21.15.52:0/3018255790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f0c144220 0x7f8f0c1446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f0a59c640 1 --2- 172.21.15.52:0/3018255790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f0c143b60 0x7f8f0c071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f09d9b640 1 -- 172.21.15.52:0/3018255790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f0c144220 msgr2=0x7f8f0c1446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f09d9b640 1 --2- 172.21.15.52:0/3018255790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f0c144220 0x7f8f0c1446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f09d9b640 1 -- 172.21.15.52:0/3018255790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f0c143b60 msgr2=0x7f8f0c071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f09d9b640 1 --2- 172.21.15.52:0/3018255790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f0c143b60 0x7f8f0c071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f09d9b640 1 -- 172.21.15.52:0/3018255790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0c14c5c0 con 0x7f8f0c156e60 2024-05-04T12:18:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f09d9b640 1 --2- 172.21.15.52:0/3018255790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f0c156e60 0x7f8f0c1572e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400e4b0 tx=0x7f8ef400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f02ffd640 1 -- 172.21.15.52:0/3018255790 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4019070 con 0x7f8f0c156e60 2024-05-04T12:18:20.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f02ffd640 1 -- 172.21.15.52:0/3018255790 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef40052e0 con 0x7f8f0c156e60 2024-05-04T12:18:20.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 -- 172.21.15.52:0/3018255790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0c071ef0 con 0x7f8f0c156e60 2024-05-04T12:18:20.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f02ffd640 1 -- 172.21.15.52:0/3018255790 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4003840 con 0x7f8f0c156e60 2024-05-04T12:18:20.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.102+0000 7f8f10d19640 1 -- 172.21.15.52:0/3018255790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f0c14ae40 con 0x7f8f0c156e60 2024-05-04T12:18:20.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.106+0000 7f8f10d19640 1 -- 172.21.15.52:0/3018255790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f0c157820 con 0x7f8f0c156e60 2024-05-04T12:18:20.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.106+0000 7f8f02ffd640 1 -- 172.21.15.52:0/3018255790 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8ef401c070 con 0x7f8f0c156e60 2024-05-04T12:18:20.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.106+0000 7f8f02ffd640 1 --2- 172.21.15.52:0/3018255790 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8eec0678c0 0x7f8eec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.106+0000 7f8f02ffd640 1 -- 172.21.15.52:0/3018255790 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f8ef4084e70 con 0x7f8f0c156e60 2024-05-04T12:18:20.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.106+0000 7f8f0a59c640 1 --2- 172.21.15.52:0/3018255790 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8eec0678c0 0x7f8eec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.106+0000 7f8f0a59c640 1 --2- 172.21.15.52:0/3018255790 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8eec0678c0 0x7f8eec069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0ab000 tx=0x7f8efc0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:20.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.110+0000 7f8f02ffd640 1 -- 172.21.15.52:0/3018255790 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8f0c14c5c0 con 0x7f8f0c156e60 2024-05-04T12:18:20.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.274+0000 7f8f10d19640 1 -- 172.21.15.52:0/3018255790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8f0c072500 con 0x7f8f0c156e60 2024-05-04T12:18:20.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.274+0000 7f8f02ffd640 1 -- 172.21.15.52:0/3018255790 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f8f0c072500 con 0x7f8f0c156e60 2024-05-04T12:18:20.275 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:20.275 INFO:teuthology.orchestra.run.smithi052.stdout:[ 2024-05-04T12:18:20.275 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:18:20.275 INFO:teuthology.orchestra.run.smithi052.stdout: "poolnum": 1, 2024-05-04T12:18:20.276 INFO:teuthology.orchestra.run.smithi052.stdout: "poolname": ".mgr" 2024-05-04T12:18:20.276 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:18:20.276 INFO:teuthology.orchestra.run.smithi052.stdout:] 2024-05-04T12:18:20.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.274+0000 7f8f10d19640 1 -- 172.21.15.52:0/3018255790 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8eec0678c0 msgr2=0x7f8eec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.274+0000 7f8f10d19640 1 --2- 172.21.15.52:0/3018255790 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8eec0678c0 0x7f8eec069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0ab000 tx=0x7f8efc0a7070 comp rx=0 tx=0).stop 2024-05-04T12:18:20.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.274+0000 7f8f10d19640 1 -- 172.21.15.52:0/3018255790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f0c156e60 msgr2=0x7f8f0c1572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.274+0000 7f8f10d19640 1 --2- 172.21.15.52:0/3018255790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f0c156e60 0x7f8f0c1572e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400e4b0 tx=0x7f8ef400e970 comp rx=0 tx=0).stop 2024-05-04T12:18:20.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.274+0000 7f8f10d19640 1 -- 172.21.15.52:0/3018255790 shutdown_connections 2024-05-04T12:18:20.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.274+0000 7f8f10d19640 1 --2- 172.21.15.52:0/3018255790 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8eec0678c0 0x7f8eec069d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.274+0000 7f8f10d19640 1 --2- 172.21.15.52:0/3018255790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f0c144220 0x7f8f0c1446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.274+0000 7f8f10d19640 1 --2- 172.21.15.52:0/3018255790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f0c156e60 0x7f8f0c1572e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.274+0000 7f8f10d19640 1 --2- 172.21.15.52:0/3018255790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f0c143b60 0x7f8f0c071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.274+0000 7f8f10d19640 1 -- 172.21.15.52:0/3018255790 >> 172.21.15.52:0/3018255790 conn(0x7f8f0c0b14a0 msgr2=0x7f8f0c1597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:20.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.274+0000 7f8f10d19640 1 -- 172.21.15.52:0/3018255790 shutdown_connections 2024-05-04T12:18:20.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.274+0000 7f8f10d19640 1 -- 172.21.15.52:0/3018255790 wait complete. 2024-05-04T12:18:20.294 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-04T12:18:20.294 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-04T12:18:20.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.422+0000 7f6a784e2640 1 Processor -- start 2024-05-04T12:18:20.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.422+0000 7f6a784e2640 1 -- start start 2024-05-04T12:18:20.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.422+0000 7f6a784e2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6a7006f560 0x7f6a70166f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.422+0000 7f6a784e2640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a700ced50 con 0x7f6a701608f0 2024-05-04T12:18:20.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.422+0000 7f6a784e2640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a7015fe10 con 0x7f6a70144fc0 2024-05-04T12:18:20.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.422+0000 7f6a784e2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f6a701453a0 con 0x7f6a7006f560 2024-05-04T12:18:20.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.422+0000 7f6a75a7d640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f6a701608f0 0x7f6a7006ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58888/0 (socket says 172.21.15.52:58888) 2024-05-04T12:18:20.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.422+0000 7f6a75a7d640 1 -- 172.21.15.52:0/3953946136 learned_addr learned my addr 172.21.15.52:0/3953946136 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:20.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.422+0000 7f6a76a7f640 1 --2- 172.21.15.52:0/3953946136 >> v2:172.21.15.111:3300/0 conn(0x7f6a7006f560 0x7f6a70166f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a7527c640 1 -- 172.21.15.52:0/3953946136 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2016419667 0 0) 0x7f6a700ced50 con 0x7f6a701608f0 2024-05-04T12:18:20.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a7527c640 1 -- 172.21.15.52:0/3953946136 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a5c003530 con 0x7f6a701608f0 2024-05-04T12:18:20.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a76a7f640 1 -- 172.21.15.52:0/3953946136 >> v1:172.21.15.111:6789/0 conn(0x7f6a70144fc0 legacy=0x7f6a700ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a76a7f640 1 -- 172.21.15.52:0/3953946136 >> v1:172.21.15.52:6789/0 conn(0x7f6a701608f0 legacy=0x7f6a7006ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a76a7f640 1 -- 172.21.15.52:0/3953946136 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a701600f0 con 0x7f6a7006f560 2024-05-04T12:18:20.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a76a7f640 1 --2- 172.21.15.52:0/3953946136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a7006f560 0x7f6a70166f50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009d90 tx=0x7f6a6801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c39bd7054a170f94 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:20.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a7527c640 1 -- 172.21.15.52:0/3953946136 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a680b2070 con 0x7f6a7006f560 2024-05-04T12:18:20.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a7527c640 1 -- 172.21.15.52:0/3953946136 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a680a5820 con 0x7f6a7006f560 2024-05-04T12:18:20.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a7527c640 1 -- 172.21.15.52:0/3953946136 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a680a5bf0 con 0x7f6a7006f560 2024-05-04T12:18:20.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a784e2640 1 -- 172.21.15.52:0/3953946136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a7006f560 msgr2=0x7f6a70166f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a784e2640 1 --2- 172.21.15.52:0/3953946136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a7006f560 0x7f6a70166f50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009d90 tx=0x7f6a6801c8e0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a784e2640 1 -- 172.21.15.52:0/3953946136 shutdown_connections 2024-05-04T12:18:20.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a784e2640 1 --2- 172.21.15.52:0/3953946136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a7006f560 0x7f6a70166f50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a784e2640 1 -- 172.21.15.52:0/3953946136 >> 172.21.15.52:0/3953946136 conn(0x7f6a70067760 msgr2=0x7f6a700b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:20.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a784e2640 1 -- 172.21.15.52:0/3953946136 shutdown_connections 2024-05-04T12:18:20.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.426+0000 7f6a784e2640 1 -- 172.21.15.52:0/3953946136 wait complete. 2024-05-04T12:18:20.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a784e2640 1 Processor -- start 2024-05-04T12:18:20.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a784e2640 1 -- start start 2024-05-04T12:18:20.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a784e2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a7006f560 0x7f6a70158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a784e2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70144fc0 0x7f6a7015e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a784e2640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a701608f0 0x7f6a7015c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a784e2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6a70169e30 con 0x7f6a7006f560 2024-05-04T12:18:20.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a784e2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6a70169cb0 con 0x7f6a70144fc0 2024-05-04T12:18:20.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a784e2640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f6a70169fb0 con 0x7f6a701608f0 2024-05-04T12:18:20.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a7627e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a7006f560 0x7f6a70158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a7627e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a7006f560 0x7f6a70158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54122/0 (socket says 172.21.15.52:54122) 2024-05-04T12:18:20.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a7627e640 1 -- 172.21.15.52:0/4158247692 learned_addr learned my addr 172.21.15.52:0/4158247692 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:20.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a75a7d640 1 --2- 172.21.15.52:0/4158247692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70144fc0 0x7f6a7015e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a76a7f640 1 --2- 172.21.15.52:0/4158247692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a701608f0 0x7f6a7015c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a7627e640 1 -- 172.21.15.52:0/4158247692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a701608f0 msgr2=0x7f6a7015c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a7627e640 1 --2- 172.21.15.52:0/4158247692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a701608f0 0x7f6a7015c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a7627e640 1 -- 172.21.15.52:0/4158247692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70144fc0 msgr2=0x7f6a7015e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a7627e640 1 --2- 172.21.15.52:0/4158247692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70144fc0 0x7f6a7015e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a7627e640 1 -- 172.21.15.52:0/4158247692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7013c460 con 0x7f6a7006f560 2024-05-04T12:18:20.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a7627e640 1 --2- 172.21.15.52:0/4158247692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a7006f560 0x7f6a70158c30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00c3a0 tx=0x7f6a6c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:20.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a66ffd640 1 -- 172.21.15.52:0/4158247692 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c0052d0 con 0x7f6a7006f560 2024-05-04T12:18:20.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a66ffd640 1 -- 172.21.15.52:0/4158247692 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a6c005470 con 0x7f6a7006f560 2024-05-04T12:18:20.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a784e2640 1 -- 172.21.15.52:0/4158247692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a7015d010 con 0x7f6a7006f560 2024-05-04T12:18:20.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a66ffd640 1 -- 172.21.15.52:0/4158247692 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c007260 con 0x7f6a7006f560 2024-05-04T12:18:20.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.430+0000 7f6a784e2640 1 -- 172.21.15.52:0/4158247692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a7013c6e0 con 0x7f6a7006f560 2024-05-04T12:18:20.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.434+0000 7f6a784e2640 1 -- 172.21.15.52:0/4158247692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a70062e50 con 0x7f6a7006f560 2024-05-04T12:18:20.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.434+0000 7f6a66ffd640 1 -- 172.21.15.52:0/4158247692 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6a6c007400 con 0x7f6a7006f560 2024-05-04T12:18:20.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.434+0000 7f6a66ffd640 1 --2- 172.21.15.52:0/4158247692 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6a540678c0 0x7f6a54069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.434+0000 7f6a75a7d640 1 --2- 172.21.15.52:0/4158247692 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6a540678c0 0x7f6a54069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.434+0000 7f6a66ffd640 1 -- 172.21.15.52:0/4158247692 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f6a6c084d60 con 0x7f6a7006f560 2024-05-04T12:18:20.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.434+0000 7f6a75a7d640 1 --2- 172.21.15.52:0/4158247692 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6a540678c0 0x7f6a54069d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6a600011f0 tx=0x7f6a60007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:20.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.438+0000 7f6a66ffd640 1 -- 172.21.15.52:0/4158247692 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6a7013c460 con 0x7f6a7006f560 2024-05-04T12:18:20.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.594+0000 7f6a784e2640 1 -- 172.21.15.52:0/4158247692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6a70075320 con 0x7f6a7006f560 2024-05-04T12:18:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.798+0000 7f6a66ffd640 1 -- 172.21.15.52:0/4158247692 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f6a6c055500 con 0x7f6a7006f560 2024-05-04T12:18:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' created 2024-05-04T12:18:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.802+0000 7f6a784e2640 1 -- 172.21.15.52:0/4158247692 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6a540678c0 msgr2=0x7f6a54069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.802+0000 7f6a784e2640 1 --2- 172.21.15.52:0/4158247692 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6a540678c0 0x7f6a54069d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6a600011f0 tx=0x7f6a60007d40 comp rx=0 tx=0).stop 2024-05-04T12:18:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.802+0000 7f6a784e2640 1 -- 172.21.15.52:0/4158247692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a7006f560 msgr2=0x7f6a70158c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.802+0000 7f6a784e2640 1 --2- 172.21.15.52:0/4158247692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a7006f560 0x7f6a70158c30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00c3a0 tx=0x7f6a6c00c860 comp rx=0 tx=0).stop 2024-05-04T12:18:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.802+0000 7f6a784e2640 1 -- 172.21.15.52:0/4158247692 shutdown_connections 2024-05-04T12:18:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.802+0000 7f6a784e2640 1 --2- 172.21.15.52:0/4158247692 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6a540678c0 0x7f6a54069d80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.802+0000 7f6a784e2640 1 --2- 172.21.15.52:0/4158247692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a701608f0 0x7f6a7015c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.802+0000 7f6a784e2640 1 --2- 172.21.15.52:0/4158247692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70144fc0 0x7f6a7015e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.802+0000 7f6a784e2640 1 --2- 172.21.15.52:0/4158247692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a7006f560 0x7f6a70158c30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.802+0000 7f6a784e2640 1 -- 172.21.15.52:0/4158247692 >> 172.21.15.52:0/4158247692 conn(0x7f6a70067760 msgr2=0x7f6a700b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.802+0000 7f6a784e2640 1 -- 172.21.15.52:0/4158247692 shutdown_connections 2024-05-04T12:18:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.802+0000 7f6a784e2640 1 -- 172.21.15.52:0/4158247692 wait complete. 2024-05-04T12:18:20.832 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-04T12:18:20.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.958+0000 7faff7186640 1 Processor -- start 2024-05-04T12:18:20.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.958+0000 7faff7186640 1 -- start start 2024-05-04T12:18:20.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.958+0000 7faff7186640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7faff015bd90 0x7faff00cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.958+0000 7faff7186640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faff006fa80 0x7faff00cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.958+0000 7faff7186640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff00ccb80 con 0x7faff015c170 2024-05-04T12:18:20.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.958+0000 7faff7186640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7faff00cc930 con 0x7faff015bd90 2024-05-04T12:18:20.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.958+0000 7faff7186640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7faff00cd4b0 con 0x7faff006fa80 2024-05-04T12:18:20.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.958+0000 7faff4f22640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7faff015bd90 0x7faff00cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.958+0000 7faff4f22640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7faff015bd90 0x7faff00cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54132/0 (socket says 172.21.15.52:54132) 2024-05-04T12:18:20.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.958+0000 7faff4f22640 1 -- 172.21.15.52:0/376142402 learned_addr learned my addr 172.21.15.52:0/376142402 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:20.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.958+0000 7faff4f22640 1 -- 172.21.15.52:0/376142402 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faff006fa80 msgr2=0x7faff00cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:20.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.958+0000 7faff4f22640 1 --2- 172.21.15.52:0/376142402 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faff006fa80 0x7faff00cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.958+0000 7faff4f22640 1 -- 172.21.15.52:0/376142402 >> v1:172.21.15.111:6789/0 conn(0x7faff015c170 legacy=0x7faff006f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:20.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.958+0000 7faff4f22640 1 -- 172.21.15.52:0/376142402 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff00cce60 con 0x7faff015bd90 2024-05-04T12:18:20.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.962+0000 7faff4f22640 1 --2- 172.21.15.52:0/376142402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faff015bd90 0x7faff00cf060 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fafec009870 tx=0x7fafec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33522e6f638532ac server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:20.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.962+0000 7fafe77fe640 1 -- 172.21.15.52:0/376142402 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec0b0070 con 0x7faff015bd90 2024-05-04T12:18:20.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.962+0000 7fafe77fe640 1 -- 172.21.15.52:0/376142402 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafec0ab600 con 0x7faff015bd90 2024-05-04T12:18:20.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.962+0000 7fafe77fe640 1 -- 172.21.15.52:0/376142402 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec0ab9d0 con 0x7faff015bd90 2024-05-04T12:18:20.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.962+0000 7faff7186640 1 -- 172.21.15.52:0/376142402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faff015bd90 msgr2=0x7faff00cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.962+0000 7faff7186640 1 --2- 172.21.15.52:0/376142402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faff015bd90 0x7faff00cf060 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fafec009870 tx=0x7fafec01c490 comp rx=0 tx=0).stop 2024-05-04T12:18:20.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.962+0000 7faff7186640 1 -- 172.21.15.52:0/376142402 shutdown_connections 2024-05-04T12:18:20.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.962+0000 7faff7186640 1 --2- 172.21.15.52:0/376142402 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faff006fa80 0x7faff00cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.962+0000 7faff7186640 1 --2- 172.21.15.52:0/376142402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faff015bd90 0x7faff00cf060 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.962+0000 7faff7186640 1 -- 172.21.15.52:0/376142402 >> 172.21.15.52:0/376142402 conn(0x7faff0067730 msgr2=0x7faff00b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:20.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.962+0000 7faff7186640 1 -- 172.21.15.52:0/376142402 shutdown_connections 2024-05-04T12:18:20.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.962+0000 7faff7186640 1 -- 172.21.15.52:0/376142402 wait complete. 2024-05-04T12:18:20.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.962+0000 7faff7186640 1 Processor -- start 2024-05-04T12:18:20.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.962+0000 7faff7186640 1 -- start start 2024-05-04T12:18:20.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7faff7186640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faff006fa80 0x7faff014be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7faff7186640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faff015bd90 0x7faff014a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7faff7186640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faff015c170 0x7faff014e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7faff7186640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faff006d820 con 0x7faff015bd90 2024-05-04T12:18:20.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7faff7186640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faff006d6a0 con 0x7faff015c170 2024-05-04T12:18:20.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7faff7186640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7faff006d9a0 con 0x7faff006fa80 2024-05-04T12:18:20.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faff015bd90 0x7faff014a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faff015bd90 0x7faff014a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54138/0 (socket says 172.21.15.52:54138) 2024-05-04T12:18:20.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe7fff640 1 -- 172.21.15.52:0/2884528104 learned_addr learned my addr 172.21.15.52:0/2884528104 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:20.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7faff5723640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faff015c170 0x7faff014e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7faff4f22640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faff006fa80 0x7faff014be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe7fff640 1 -- 172.21.15.52:0/2884528104 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faff006fa80 msgr2=0x7faff014be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe7fff640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faff006fa80 0x7faff014be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe7fff640 1 -- 172.21.15.52:0/2884528104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faff015c170 msgr2=0x7faff014e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:20.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe7fff640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faff015c170 0x7faff014e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:20.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe7fff640 1 -- 172.21.15.52:0/2884528104 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff014ea60 con 0x7faff015bd90 2024-05-04T12:18:20.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7faff4f22640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faff006fa80 0x7faff014be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:20.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7faff5723640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faff015c170 0x7faff014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:20.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe7fff640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faff015bd90 0x7faff014a730 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fafe000c490 tx=0x7fafe0001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:20.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe57fa640 1 -- 172.21.15.52:0/2884528104 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0017070 con 0x7faff015bd90 2024-05-04T12:18:20.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe57fa640 1 -- 172.21.15.52:0/2884528104 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe0003600 con 0x7faff015bd90 2024-05-04T12:18:20.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe57fa640 1 -- 172.21.15.52:0/2884528104 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0003a30 con 0x7faff015bd90 2024-05-04T12:18:20.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7faff7186640 1 -- 172.21.15.52:0/2884528104 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff015ab40 con 0x7faff015bd90 2024-05-04T12:18:20.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7faff7186640 1 -- 172.21.15.52:0/2884528104 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff014dd20 con 0x7faff015bd90 2024-05-04T12:18:20.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7faff7186640 1 -- 172.21.15.52:0/2884528104 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff0062e50 con 0x7faff015bd90 2024-05-04T12:18:20.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe57fa640 1 -- 172.21.15.52:0/2884528104 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fafe0003bd0 con 0x7faff015bd90 2024-05-04T12:18:20.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe57fa640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fafdc0678c0 0x7fafdc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:20.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7fafe57fa640 1 -- 172.21.15.52:0/2884528104 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fafe0084e60 con 0x7faff015bd90 2024-05-04T12:18:20.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.966+0000 7faff4f22640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fafdc0678c0 0x7fafdc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:20.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.970+0000 7faff4f22640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fafdc0678c0 0x7fafdc069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fafec0ad000 tx=0x7fafec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:20.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:20.974+0000 7fafe57fa640 1 -- 172.21.15.52:0/2884528104 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fafe0055a60 con 0x7faff015bd90 2024-05-04T12:18:21.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.134+0000 7faff7186640 1 -- 172.21.15.52:0/2884528104 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faff0075320 con 0x7faff015bd90 2024-05-04T12:18:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.802+0000 7fafe57fa640 1 -- 172.21.15.52:0/2884528104 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7fafe0055360 con 0x7faff015bd90 2024-05-04T12:18:21.807 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' created 2024-05-04T12:18:21.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.810+0000 7faff7186640 1 -- 172.21.15.52:0/2884528104 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fafdc0678c0 msgr2=0x7fafdc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:21.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.810+0000 7faff7186640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fafdc0678c0 0x7fafdc069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fafec0ad000 tx=0x7fafec008040 comp rx=0 tx=0).stop 2024-05-04T12:18:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.810+0000 7faff7186640 1 -- 172.21.15.52:0/2884528104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faff015bd90 msgr2=0x7faff014a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.810+0000 7faff7186640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faff015bd90 0x7faff014a730 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fafe000c490 tx=0x7fafe0001e80 comp rx=0 tx=0).stop 2024-05-04T12:18:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.810+0000 7faff7186640 1 -- 172.21.15.52:0/2884528104 shutdown_connections 2024-05-04T12:18:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.810+0000 7faff7186640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fafdc0678c0 0x7fafdc069d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.810+0000 7faff7186640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faff015c170 0x7faff014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.810+0000 7faff7186640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faff015bd90 0x7faff014a730 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.810+0000 7faff7186640 1 --2- 172.21.15.52:0/2884528104 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faff006fa80 0x7faff014be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.810+0000 7faff7186640 1 -- 172.21.15.52:0/2884528104 >> 172.21.15.52:0/2884528104 conn(0x7faff0067730 msgr2=0x7faff00b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.810+0000 7faff7186640 1 -- 172.21.15.52:0/2884528104 shutdown_connections 2024-05-04T12:18:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.810+0000 7faff7186640 1 -- 172.21.15.52:0/2884528104 wait complete. 2024-05-04T12:18:21.848 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-04T12:18:21.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408c5a5640 1 Processor -- start 2024-05-04T12:18:21.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408c5a5640 1 -- start start 2024-05-04T12:18:21.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408c5a5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f408415d400 0x7f408413f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:21.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408c5a5640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f408406c6c0 0x7f408406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:21.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408c5a5640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4084160670 con 0x7f408415d020 2024-05-04T12:18:21.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408c5a5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f408413f6a0 con 0x7f408415d400 2024-05-04T12:18:21.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408c5a5640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f40840a9480 con 0x7f408406c6c0 2024-05-04T12:18:21.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408a341640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f408415d020 0x7f408415fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58896/0 (socket says 172.21.15.52:58896) 2024-05-04T12:18:21.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408a341640 1 -- 172.21.15.52:0/3873251535 learned_addr learned my addr 172.21.15.52:0/3873251535 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:21.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408ab42640 1 --2- 172.21.15.52:0/3873251535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f408406c6c0 0x7f408406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:21.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f4089b40640 1 --2- 172.21.15.52:0/3873251535 >> v2:172.21.15.111:3300/0 conn(0x7f408415d400 0x7f408413f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:21.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408933f640 1 -- 172.21.15.52:0/3873251535 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1994112355 0 0) 0x7f4084160670 con 0x7f408415d020 2024-05-04T12:18:21.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408933f640 1 -- 172.21.15.52:0/3873251535 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4070003530 con 0x7f408415d020 2024-05-04T12:18:21.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408ab42640 1 -- 172.21.15.52:0/3873251535 >> v2:172.21.15.111:3300/0 conn(0x7f408415d400 msgr2=0x7f408413f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:21.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408ab42640 1 --2- 172.21.15.52:0/3873251535 >> v2:172.21.15.111:3300/0 conn(0x7f408415d400 0x7f408413f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:21.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408933f640 1 -- 172.21.15.52:0/3873251535 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2725048697 0 0) 0x7f4070003530 con 0x7f408415d020 2024-05-04T12:18:21.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408ab42640 1 -- 172.21.15.52:0/3873251535 >> v1:172.21.15.52:6789/0 conn(0x7f408415d020 legacy=0x7f408415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:21.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f408ab42640 1 -- 172.21.15.52:0/3873251535 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4084160950 con 0x7f408406c6c0 2024-05-04T12:18:21.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.974+0000 7f4089b40640 1 --2- 172.21.15.52:0/3873251535 >> v2:172.21.15.111:3300/0 conn(0x7f408415d400 0x7f408413f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:18:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.978+0000 7f408ab42640 1 --2- 172.21.15.52:0/3873251535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f408406c6c0 0x7f408406cb50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f407c009d90 tx=0x7f407c01c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0763cf5839dc1c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.978+0000 7f408933f640 1 -- 172.21.15.52:0/3873251535 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c0ac070 con 0x7f408406c6c0 2024-05-04T12:18:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.978+0000 7f408933f640 1 -- 172.21.15.52:0/3873251535 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f407c0a6410 con 0x7f408406c6c0 2024-05-04T12:18:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.978+0000 7f408933f640 1 -- 172.21.15.52:0/3873251535 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4074001680 con 0x7f408406c6c0 2024-05-04T12:18:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.978+0000 7f408c5a5640 1 -- 172.21.15.52:0/3873251535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f408406c6c0 msgr2=0x7f408406cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.978+0000 7f408c5a5640 1 --2- 172.21.15.52:0/3873251535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f408406c6c0 0x7f408406cb50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f407c009d90 tx=0x7f407c01c800 comp rx=0 tx=0).stop 2024-05-04T12:18:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.978+0000 7f408c5a5640 1 -- 172.21.15.52:0/3873251535 shutdown_connections 2024-05-04T12:18:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.978+0000 7f408c5a5640 1 --2- 172.21.15.52:0/3873251535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f408406c6c0 0x7f408406cb50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.978+0000 7f408c5a5640 1 --2- 172.21.15.52:0/3873251535 >> v2:172.21.15.111:3300/0 conn(0x7f408415d400 0x7f408413f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.978+0000 7f408c5a5640 1 -- 172.21.15.52:0/3873251535 >> 172.21.15.52:0/3873251535 conn(0x7f408406b210 msgr2=0x7f408406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.978+0000 7f408c5a5640 1 -- 172.21.15.52:0/3873251535 shutdown_connections 2024-05-04T12:18:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.978+0000 7f408c5a5640 1 -- 172.21.15.52:0/3873251535 wait complete. 2024-05-04T12:18:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408c5a5640 1 Processor -- start 2024-05-04T12:18:21.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408c5a5640 1 -- start start 2024-05-04T12:18:21.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408c5a5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f408406c6c0 0x7f4084138c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408c5a5640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f408415d020 0x7f40841519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408c5a5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f408415d400 0x7f40841523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408c5a5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40840aa580 con 0x7f408415d400 2024-05-04T12:18:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408c5a5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f40840aa400 con 0x7f408406c6c0 2024-05-04T12:18:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408c5a5640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f40840aa700 con 0x7f408415d020 2024-05-04T12:18:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408ab42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f408415d400 0x7f40841523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408ab42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f408415d400 0x7f40841523c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54148/0 (socket says 172.21.15.52:54148) 2024-05-04T12:18:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408ab42640 1 -- 172.21.15.52:0/4249479621 learned_addr learned my addr 172.21.15.52:0/4249479621 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f4089b40640 1 --2- 172.21.15.52:0/4249479621 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f408415d020 0x7f40841519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408a341640 1 --2- 172.21.15.52:0/4249479621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f408406c6c0 0x7f4084138c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408ab42640 1 -- 172.21.15.52:0/4249479621 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f408415d020 msgr2=0x7f40841519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408ab42640 1 --2- 172.21.15.52:0/4249479621 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f408415d020 0x7f40841519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408ab42640 1 -- 172.21.15.52:0/4249479621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f408406c6c0 msgr2=0x7f4084138c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408ab42640 1 --2- 172.21.15.52:0/4249479621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f408406c6c0 0x7f4084138c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408ab42640 1 -- 172.21.15.52:0/4249479621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408415c610 con 0x7f408415d400 2024-05-04T12:18:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408ab42640 1 --2- 172.21.15.52:0/4249479621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f408415d400 0x7f40841523c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f407c009d90 tx=0x7f407c0195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f407affd640 1 -- 172.21.15.52:0/4249479621 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c0ac070 con 0x7f408415d400 2024-05-04T12:18:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f407affd640 1 -- 172.21.15.52:0/4249479621 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f407c004450 con 0x7f408415d400 2024-05-04T12:18:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408c5a5640 1 -- 172.21.15.52:0/4249479621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4084158170 con 0x7f408415d400 2024-05-04T12:18:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f407affd640 1 -- 172.21.15.52:0/4249479621 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c004800 con 0x7f408415d400 2024-05-04T12:18:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408c5a5640 1 -- 172.21.15.52:0/4249479621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4084158410 con 0x7f408415d400 2024-05-04T12:18:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f408c5a5640 1 -- 172.21.15.52:0/4249479621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4084062e50 con 0x7f408415d400 2024-05-04T12:18:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.982+0000 7f407affd640 1 -- 172.21.15.52:0/4249479621 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f407c0af070 con 0x7f408415d400 2024-05-04T12:18:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.986+0000 7f407affd640 1 --2- 172.21.15.52:0/4249479621 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f40680678c0 0x7f4068069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.986+0000 7f407affd640 1 -- 172.21.15.52:0/4249479621 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f407c11fd90 con 0x7f408415d400 2024-05-04T12:18:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.986+0000 7f408a341640 1 --2- 172.21.15.52:0/4249479621 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f40680678c0 0x7f4068069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.986+0000 7f408a341640 1 --2- 172.21.15.52:0/4249479621 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f40680678c0 0x7f4068069d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f408413efc0 tx=0x7f4074009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:21.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:21.990+0000 7f407affd640 1 -- 172.21.15.52:0/4249479621 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f407c0c1030 con 0x7f408415d400 2024-05-04T12:18:22.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.182+0000 7f408c5a5640 1 -- 172.21.15.52:0/4249479621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4084075320 con 0x7f408415d400 2024-05-04T12:18:22.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.822+0000 7f407affd640 1 -- 172.21.15.52:0/4249479621 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f407c016480 con 0x7f408415d400 2024-05-04T12:18:22.824 INFO:teuthology.orchestra.run.smithi052.stderr:new fs with metadata pool 2 and data pool 3 2024-05-04T12:18:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.826+0000 7f408c5a5640 1 -- 172.21.15.52:0/4249479621 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f40680678c0 msgr2=0x7f4068069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.826+0000 7f408c5a5640 1 --2- 172.21.15.52:0/4249479621 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f40680678c0 0x7f4068069d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f408413efc0 tx=0x7f4074009040 comp rx=0 tx=0).stop 2024-05-04T12:18:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.826+0000 7f408c5a5640 1 -- 172.21.15.52:0/4249479621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f408415d400 msgr2=0x7f40841523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.826+0000 7f408c5a5640 1 --2- 172.21.15.52:0/4249479621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f408415d400 0x7f40841523c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f407c009d90 tx=0x7f407c0195a0 comp rx=0 tx=0).stop 2024-05-04T12:18:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.826+0000 7f408c5a5640 1 -- 172.21.15.52:0/4249479621 shutdown_connections 2024-05-04T12:18:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.826+0000 7f408c5a5640 1 --2- 172.21.15.52:0/4249479621 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f40680678c0 0x7f4068069d80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.826+0000 7f408c5a5640 1 --2- 172.21.15.52:0/4249479621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f408415d400 0x7f40841523c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.826+0000 7f408c5a5640 1 --2- 172.21.15.52:0/4249479621 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f408415d020 0x7f40841519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.826+0000 7f408c5a5640 1 --2- 172.21.15.52:0/4249479621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f408406c6c0 0x7f4084138c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.826+0000 7f408c5a5640 1 -- 172.21.15.52:0/4249479621 >> 172.21.15.52:0/4249479621 conn(0x7f408406b210 msgr2=0x7f4084140070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.826+0000 7f408c5a5640 1 -- 172.21.15.52:0/4249479621 shutdown_connections 2024-05-04T12:18:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.826+0000 7f408c5a5640 1 -- 172.21.15.52:0/4249479621 wait complete. 2024-05-04T12:18:22.867 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:22.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.994+0000 7fa3a3e45640 1 Processor -- start 2024-05-04T12:18:22.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.994+0000 7fa3a3e45640 1 -- start start 2024-05-04T12:18:23.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a3e45640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa39c0b6bf0 0x7fa39c169130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a3e45640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa39c15eac0 con 0x7fa39c144590 2024-05-04T12:18:23.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a3e45640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa39c15eda0 con 0x7fa39c1441b0 2024-05-04T12:18:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a3e45640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fa39c15f6d0 con 0x7fa39c0b6bf0 2024-05-04T12:18:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a13e0640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa39c144590 0x7fa39c0b64e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58908/0 (socket says 172.21.15.52:58908) 2024-05-04T12:18:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a13e0640 1 -- 172.21.15.52:0/4140086488 learned_addr learned my addr 172.21.15.52:0/4140086488 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a23e2640 1 --2- 172.21.15.52:0/4140086488 >> v2:172.21.15.111:3300/0 conn(0x7fa39c0b6bf0 0x7fa39c169130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 -- 172.21.15.52:0/4140086488 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3418992880 0 0) 0x7fa39c15eac0 con 0x7fa39c144590 2024-05-04T12:18:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 -- 172.21.15.52:0/4140086488 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa388003530 con 0x7fa39c144590 2024-05-04T12:18:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 -- 172.21.15.52:0/4140086488 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2344098535 0 0) 0x7fa388003530 con 0x7fa39c144590 2024-05-04T12:18:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 -- 172.21.15.52:0/4140086488 >> v2:172.21.15.111:3300/0 conn(0x7fa39c0b6bf0 msgr2=0x7fa39c169130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 --2- 172.21.15.52:0/4140086488 >> v2:172.21.15.111:3300/0 conn(0x7fa39c0b6bf0 0x7fa39c169130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 -- 172.21.15.52:0/4140086488 >> v1:172.21.15.111:6789/0 conn(0x7fa39c1441b0 legacy=0x7fa39c1627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 -- 172.21.15.52:0/4140086488 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39c15f080 con 0x7fa39c144590 2024-05-04T12:18:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 -- 172.21.15.52:0/4140086488 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7fa38c001680 con 0x7fa39c144590 2024-05-04T12:18:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 -- 172.21.15.52:0/4140086488 >> v1:172.21.15.52:6789/0 conn(0x7fa39c144590 legacy=0x7fa39c0b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 --2- 172.21.15.52:0/4140086488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3880039c0 0x7fa388003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 --2- 172.21.15.52:0/4140086488 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa388004810 0x7fa388004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 --2- 172.21.15.52:0/4140086488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3880051d0 0x7fa38800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 -- 172.21.15.52:0/4140086488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c15f6d0 con 0x7fa3880039c0 2024-05-04T12:18:23.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 -- 172.21.15.52:0/4140086488 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa38800c0a0 con 0x7fa3880051d0 2024-05-04T12:18:23.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a0bdf640 1 -- 172.21.15.52:0/4140086488 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa38800c2a0 con 0x7fa388004810 2024-05-04T12:18:23.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a1be1640 1 --2- 172.21.15.52:0/4140086488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3880039c0 0x7fa388003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a13e0640 1 --2- 172.21.15.52:0/4140086488 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa388004810 0x7fa388004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a23e2640 1 --2- 172.21.15.52:0/4140086488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3880051d0 0x7fa38800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a1be1640 1 -- 172.21.15.52:0/4140086488 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa388004810 msgr2=0x7fa388004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:23.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a1be1640 1 --2- 172.21.15.52:0/4140086488 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa388004810 0x7fa388004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a1be1640 1 -- 172.21.15.52:0/4140086488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3880051d0 msgr2=0x7fa38800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:23.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a1be1640 1 --2- 172.21.15.52:0/4140086488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3880051d0 0x7fa38800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a1be1640 1 -- 172.21.15.52:0/4140086488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa388003530 con 0x7fa3880039c0 2024-05-04T12:18:23.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:22.998+0000 7fa3a23e2640 1 -- 172.21.15.52:0/4140086488 reap_dead start 2024-05-04T12:18:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a1be1640 1 --2- 172.21.15.52:0/4140086488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3880039c0 0x7fa388003e00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa398001780 tx=0x7fa39801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5311feb838c8a37 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a0bdf640 1 -- 172.21.15.52:0/4140086488 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3980013a0 con 0x7fa3880039c0 2024-05-04T12:18:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a0bdf640 1 -- 172.21.15.52:0/4140086488 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3980052f0 con 0x7fa3880039c0 2024-05-04T12:18:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a0bdf640 1 -- 172.21.15.52:0/4140086488 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3980a4e60 con 0x7fa3880039c0 2024-05-04T12:18:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 -- 172.21.15.52:0/4140086488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3880039c0 msgr2=0x7fa388003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 --2- 172.21.15.52:0/4140086488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3880039c0 0x7fa388003e00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa398001780 tx=0x7fa39801c3d0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 -- 172.21.15.52:0/4140086488 shutdown_connections 2024-05-04T12:18:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 --2- 172.21.15.52:0/4140086488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3880039c0 0x7fa388003e00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 -- 172.21.15.52:0/4140086488 >> 172.21.15.52:0/4140086488 conn(0x7fa39c0b52d0 msgr2=0x7fa39c0b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 -- 172.21.15.52:0/4140086488 shutdown_connections 2024-05-04T12:18:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 -- 172.21.15.52:0/4140086488 wait complete. 2024-05-04T12:18:23.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 Processor -- start 2024-05-04T12:18:23.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 -- start start 2024-05-04T12:18:23.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3880039c0 0x7fa39c1534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa39c156ad0 0x7fa39c156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa39c157d00 0x7fa39c158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c162bc0 con 0x7fa39c157d00 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c143a90 con 0x7fa39c156ad0 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a3e45640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa39c1e9e20 con 0x7fa3880039c0 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a23e2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa39c157d00 0x7fa39c158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a23e2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa39c157d00 0x7fa39c158180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54164/0 (socket says 172.21.15.52:54164) 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a23e2640 1 -- 172.21.15.52:0/466155309 learned_addr learned my addr 172.21.15.52:0/466155309 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a1be1640 1 --2- 172.21.15.52:0/466155309 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3880039c0 0x7fa39c1534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a13e0640 1 --2- 172.21.15.52:0/466155309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa39c156ad0 0x7fa39c156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a23e2640 1 -- 172.21.15.52:0/466155309 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3880039c0 msgr2=0x7fa39c1534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a23e2640 1 --2- 172.21.15.52:0/466155309 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3880039c0 0x7fa39c1534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a23e2640 1 -- 172.21.15.52:0/466155309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa39c156ad0 msgr2=0x7fa39c156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a23e2640 1 --2- 172.21.15.52:0/466155309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa39c156ad0 0x7fa39c156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a23e2640 1 -- 172.21.15.52:0/466155309 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39c1ea070 con 0x7fa39c157d00 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.002+0000 7fa3a23e2640 1 --2- 172.21.15.52:0/466155309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa39c157d00 0x7fa39c158180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0b34e0 tx=0x7fa3940115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:23.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.006+0000 7fa3927fc640 1 -- 172.21.15.52:0/466155309 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394011db0 con 0x7fa39c157d00 2024-05-04T12:18:23.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.006+0000 7fa3927fc640 1 -- 172.21.15.52:0/466155309 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa39401b070 con 0x7fa39c157d00 2024-05-04T12:18:23.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.006+0000 7fa3a3e45640 1 -- 172.21.15.52:0/466155309 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39c1ea350 con 0x7fa39c157d00 2024-05-04T12:18:23.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.006+0000 7fa3927fc640 1 -- 172.21.15.52:0/466155309 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3940039a0 con 0x7fa39c157d00 2024-05-04T12:18:23.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.006+0000 7fa3a3e45640 1 -- 172.21.15.52:0/466155309 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39c1ea910 con 0x7fa39c157d00 2024-05-04T12:18:23.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.006+0000 7fa3a3e45640 1 -- 172.21.15.52:0/466155309 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa39c062e50 con 0x7fa39c157d00 2024-05-04T12:18:23.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.006+0000 7fa3927fc640 1 -- 172.21.15.52:0/466155309 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa394003b40 con 0x7fa39c157d00 2024-05-04T12:18:23.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.006+0000 7fa3927fc640 1 --2- 172.21.15.52:0/466155309 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa3800678f0 0x7fa380069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.006+0000 7fa3927fc640 1 -- 172.21.15.52:0/466155309 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3940863e0 con 0x7fa39c157d00 2024-05-04T12:18:23.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.006+0000 7fa3a1be1640 1 --2- 172.21.15.52:0/466155309 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa3800678f0 0x7fa380069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.006+0000 7fa3a1be1640 1 --2- 172.21.15.52:0/466155309 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa3800678f0 0x7fa380069db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa3980ac000 tx=0x7fa3980011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:23.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.010+0000 7fa3927fc640 1 -- 172.21.15.52:0/466155309 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa394056f20 con 0x7fa39c157d00 2024-05-04T12:18:23.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.174+0000 7fa3a3e45640 1 -- 172.21.15.52:0/466155309 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa39c141550 con 0x7fa39c157d00 2024-05-04T12:18:23.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.178+0000 7fa3927fc640 1 -- 172.21.15.52:0/466155309 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16630 (secure 0 0 0) 0x7fa394056820 con 0x7fa39c157d00 2024-05-04T12:18:23.181 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:23.181 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":20,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:22.188976+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:23.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.182+0000 7fa3a3e45640 1 -- 172.21.15.52:0/466155309 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa3800678f0 msgr2=0x7fa380069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.182+0000 7fa3a3e45640 1 --2- 172.21.15.52:0/466155309 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa3800678f0 0x7fa380069db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa3980ac000 tx=0x7fa3980011a0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.182+0000 7fa3a3e45640 1 -- 172.21.15.52:0/466155309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa39c157d00 msgr2=0x7fa39c158180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.182+0000 7fa3a3e45640 1 --2- 172.21.15.52:0/466155309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa39c157d00 0x7fa39c158180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0b34e0 tx=0x7fa3940115a0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.182+0000 7fa3a3e45640 1 -- 172.21.15.52:0/466155309 shutdown_connections 2024-05-04T12:18:23.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.182+0000 7fa3a3e45640 1 --2- 172.21.15.52:0/466155309 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa3800678f0 0x7fa380069db0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.182+0000 7fa3a3e45640 1 --2- 172.21.15.52:0/466155309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa39c157d00 0x7fa39c158180 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.182+0000 7fa3a3e45640 1 --2- 172.21.15.52:0/466155309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa39c156ad0 0x7fa39c156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.182+0000 7fa3a3e45640 1 --2- 172.21.15.52:0/466155309 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3880039c0 0x7fa39c1534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.182+0000 7fa3a3e45640 1 -- 172.21.15.52:0/466155309 >> 172.21.15.52:0/466155309 conn(0x7fa39c0b52d0 msgr2=0x7fa39c160590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:23.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.182+0000 7fa3a3e45640 1 -- 172.21.15.52:0/466155309 shutdown_connections 2024-05-04T12:18:23.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.182+0000 7fa3a3e45640 1 -- 172.21.15.52:0/466155309 wait complete. 2024-05-04T12:18:23.200 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:23.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.314+0000 7f9184fee640 1 Processor -- start 2024-05-04T12:18:23.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f9184fee640 1 -- start start 2024-05-04T12:18:23.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f9184fee640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f91800ccdb0 0x7f91800723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f9184fee640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f91800cd190 0x7f918015c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f9184fee640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9180072aa0 con 0x7f918015c7f0 2024-05-04T12:18:23.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f9184fee640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f9180070690 con 0x7f91800ccdb0 2024-05-04T12:18:23.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f9184fee640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f91800708e0 con 0x7f91800cd190 2024-05-04T12:18:23.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f917e59c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f91800ccdb0 0x7f91800723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f917dd9b640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f918015c7f0 0x7f9180158640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58924/0 (socket says 172.21.15.52:58924) 2024-05-04T12:18:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f917dd9b640 1 -- 172.21.15.52:0/2943984514 learned_addr learned my addr 172.21.15.52:0/2943984514 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f917ed9d640 1 --2- 172.21.15.52:0/2943984514 >> v2:172.21.15.111:3300/0 conn(0x7f91800cd190 0x7f918015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f917d59a640 1 -- 172.21.15.52:0/2943984514 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2966407212 0 0) 0x7f9180072aa0 con 0x7f918015c7f0 2024-05-04T12:18:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f917e59c640 1 -- 172.21.15.52:0/2943984514 >> v2:172.21.15.111:3300/0 conn(0x7f91800cd190 msgr2=0x7f918015c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f917e59c640 1 --2- 172.21.15.52:0/2943984514 >> v2:172.21.15.111:3300/0 conn(0x7f91800cd190 0x7f918015c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f917e59c640 1 -- 172.21.15.52:0/2943984514 >> v1:172.21.15.52:6789/0 conn(0x7f918015c7f0 legacy=0x7f9180158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.318+0000 7f917e59c640 1 -- 172.21.15.52:0/2943984514 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9180070bc0 con 0x7f91800ccdb0 2024-05-04T12:18:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f917e59c640 1 --2- 172.21.15.52:0/2943984514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91800ccdb0 0x7f91800723e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f918015eb30 tx=0x7f917001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295182052718d220 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f917d59a640 1 -- 172.21.15.52:0/2943984514 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91700b0070 con 0x7f91800ccdb0 2024-05-04T12:18:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f917d59a640 1 -- 172.21.15.52:0/2943984514 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91700ab600 con 0x7f91800ccdb0 2024-05-04T12:18:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f917d59a640 1 -- 172.21.15.52:0/2943984514 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91700ab9d0 con 0x7f91800ccdb0 2024-05-04T12:18:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 -- 172.21.15.52:0/2943984514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91800ccdb0 msgr2=0x7f91800723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 --2- 172.21.15.52:0/2943984514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91800ccdb0 0x7f91800723e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f918015eb30 tx=0x7f917001c3f0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 -- 172.21.15.52:0/2943984514 shutdown_connections 2024-05-04T12:18:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 --2- 172.21.15.52:0/2943984514 >> v2:172.21.15.111:3300/0 conn(0x7f91800cd190 0x7f918015c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 --2- 172.21.15.52:0/2943984514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91800ccdb0 0x7f91800723e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 -- 172.21.15.52:0/2943984514 >> 172.21.15.52:0/2943984514 conn(0x7f91800b14a0 msgr2=0x7f91800b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 -- 172.21.15.52:0/2943984514 shutdown_connections 2024-05-04T12:18:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 -- 172.21.15.52:0/2943984514 wait complete. 2024-05-04T12:18:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 Processor -- start 2024-05-04T12:18:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 -- start start 2024-05-04T12:18:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91800ccdb0 0x7f918013d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91800cd190 0x7f918015dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f918015c7f0 0x7f918014e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9180168a10 con 0x7f91800ccdb0 2024-05-04T12:18:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9180168890 con 0x7f91800cd190 2024-05-04T12:18:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f9184fee640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f9180168b90 con 0x7f918015c7f0 2024-05-04T12:18:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f917e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91800ccdb0 0x7f918013d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f917e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91800ccdb0 0x7f918013d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54180/0 (socket says 172.21.15.52:54180) 2024-05-04T12:18:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.322+0000 7f917e59c640 1 -- 172.21.15.52:0/1512958885 learned_addr learned my addr 172.21.15.52:0/1512958885 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f917ed9d640 1 --2- 172.21.15.52:0/1512958885 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f918015c7f0 0x7f918014e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f917dd9b640 1 --2- 172.21.15.52:0/1512958885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91800cd190 0x7f918015dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f917e59c640 1 -- 172.21.15.52:0/1512958885 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f918015c7f0 msgr2=0x7f918014e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f917e59c640 1 --2- 172.21.15.52:0/1512958885 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f918015c7f0 0x7f918014e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f917e59c640 1 -- 172.21.15.52:0/1512958885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91800cd190 msgr2=0x7f918015dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f917e59c640 1 --2- 172.21.15.52:0/1512958885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91800cd190 0x7f918015dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f917e59c640 1 -- 172.21.15.52:0/1512958885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918015e160 con 0x7f91800ccdb0 2024-05-04T12:18:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f917e59c640 1 --2- 172.21.15.52:0/1512958885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91800ccdb0 0x7f918013d8d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f91700ae040 tx=0x7f917001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f916effd640 1 -- 172.21.15.52:0/1512958885 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91700b0050 con 0x7f91800ccdb0 2024-05-04T12:18:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f916effd640 1 -- 172.21.15.52:0/1512958885 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91700bc070 con 0x7f91800ccdb0 2024-05-04T12:18:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f9184fee640 1 -- 172.21.15.52:0/1512958885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918013df30 con 0x7f91800ccdb0 2024-05-04T12:18:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f916effd640 1 -- 172.21.15.52:0/1512958885 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91700aabe0 con 0x7f91800ccdb0 2024-05-04T12:18:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f9184fee640 1 -- 172.21.15.52:0/1512958885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918013e770 con 0x7f91800ccdb0 2024-05-04T12:18:23.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f9184fee640 1 -- 172.21.15.52:0/1512958885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9180062e50 con 0x7f91800ccdb0 2024-05-04T12:18:23.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f916effd640 1 -- 172.21.15.52:0/1512958885 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f91700c1460 con 0x7f91800ccdb0 2024-05-04T12:18:23.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f916effd640 1 --2- 172.21.15.52:0/1512958885 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f91600678c0 0x7f9160069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f916effd640 1 -- 172.21.15.52:0/1512958885 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9170121150 con 0x7f91800ccdb0 2024-05-04T12:18:23.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f917dd9b640 1 --2- 172.21.15.52:0/1512958885 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f91600678c0 0x7f9160069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.326+0000 7f917dd9b640 1 --2- 172.21.15.52:0/1512958885 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f91600678c0 0x7f9160069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f918015d910 tx=0x7f9174007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:23.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.330+0000 7f916effd640 1 -- 172.21.15.52:0/1512958885 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f918013df30 con 0x7f91800ccdb0 2024-05-04T12:18:23.361 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:18:23.363 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:18:23.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.486+0000 7f9184fee640 1 -- 172.21.15.52:0/1512958885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9180075320 con 0x7f91800ccdb0 2024-05-04T12:18:23.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.490+0000 7f916effd640 1 -- 172.21.15.52:0/1512958885 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16630 (secure 0 0 0) 0x7f9170016070 con 0x7f91800ccdb0 2024-05-04T12:18:23.492 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:23.492 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":20,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:22.188976+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.490+0000 7f9184fee640 1 -- 172.21.15.52:0/1512958885 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f91600678c0 msgr2=0x7f9160069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.490+0000 7f9184fee640 1 --2- 172.21.15.52:0/1512958885 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f91600678c0 0x7f9160069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f918015d910 tx=0x7f9174007b60 comp rx=0 tx=0).stop 2024-05-04T12:18:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.490+0000 7f9184fee640 1 -- 172.21.15.52:0/1512958885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91800ccdb0 msgr2=0x7f918013d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.490+0000 7f9184fee640 1 --2- 172.21.15.52:0/1512958885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91800ccdb0 0x7f918013d8d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f91700ae040 tx=0x7f917001ac30 comp rx=0 tx=0).stop 2024-05-04T12:18:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.490+0000 7f9184fee640 1 -- 172.21.15.52:0/1512958885 shutdown_connections 2024-05-04T12:18:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.490+0000 7f9184fee640 1 --2- 172.21.15.52:0/1512958885 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f91600678c0 0x7f9160069d80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.490+0000 7f9184fee640 1 --2- 172.21.15.52:0/1512958885 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f918015c7f0 0x7f918014e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.490+0000 7f9184fee640 1 --2- 172.21.15.52:0/1512958885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91800cd190 0x7f918015dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.490+0000 7f9184fee640 1 --2- 172.21.15.52:0/1512958885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91800ccdb0 0x7f918013d8d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.490+0000 7f9184fee640 1 -- 172.21.15.52:0/1512958885 >> 172.21.15.52:0/1512958885 conn(0x7f91800b14a0 msgr2=0x7f91800ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:23.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.490+0000 7f9184fee640 1 -- 172.21.15.52:0/1512958885 shutdown_connections 2024-05-04T12:18:23.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.490+0000 7f9184fee640 1 -- 172.21.15.52:0/1512958885 wait complete. 2024-05-04T12:18:23.510 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-04T12:18:23.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.630+0000 7f1072f40640 1 Processor -- start 2024-05-04T12:18:23.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.630+0000 7f1072f40640 1 -- start start 2024-05-04T12:18:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f1072f40640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f106c06eba0 0x7f106c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f1072f40640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f106c0ce190 0x7f106c0ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f1072f40640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f106c06f110 con 0x7f106c06cc40 2024-05-04T12:18:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f1072f40640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f106c06f3f0 con 0x7f106c06eba0 2024-05-04T12:18:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f1072f40640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f106c06f6d0 con 0x7f106c0ce190 2024-05-04T12:18:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f10714dd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f106c0ce190 0x7f106c0ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f10714dd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f106c0ce190 0x7f106c0ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.52:51578/0 (socket says 172.21.15.52:51578) 2024-05-04T12:18:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f10714dd640 1 -- 172.21.15.52:0/4067176507 learned_addr learned my addr 172.21.15.52:0/4067176507 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f106bfff640 1 --2- 172.21.15.52:0/4067176507 >> v2:172.21.15.111:3300/0 conn(0x7f106c06eba0 0x7f106c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f10714dd640 1 -- 172.21.15.52:0/4067176507 >> v2:172.21.15.111:3300/0 conn(0x7f106c06eba0 msgr2=0x7f106c15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f10714dd640 1 --2- 172.21.15.52:0/4067176507 >> v2:172.21.15.111:3300/0 conn(0x7f106c06eba0 0x7f106c15b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f10714dd640 1 -- 172.21.15.52:0/4067176507 >> v1:172.21.15.111:6789/0 conn(0x7f106c06cc40 legacy=0x7f106c06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f10714dd640 1 -- 172.21.15.52:0/4067176507 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106c06f9b0 con 0x7f106c0ce190 2024-05-04T12:18:23.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f10714dd640 1 --2- 172.21.15.52:0/4067176507 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f106c0ce190 0x7f106c0ce600 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f106400d150 tx=0x7f106401ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e406456b1f6ed6d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:23.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f106b7fe640 1 -- 172.21.15.52:0/4067176507 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106400b2d0 con 0x7f106c0ce190 2024-05-04T12:18:23.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f106b7fe640 1 -- 172.21.15.52:0/4067176507 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106400b470 con 0x7f106c0ce190 2024-05-04T12:18:23.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.634+0000 7f106b7fe640 1 -- 172.21.15.52:0/4067176507 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10640a7bb0 con 0x7f106c0ce190 2024-05-04T12:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 -- 172.21.15.52:0/4067176507 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f106c0ce190 msgr2=0x7f106c0ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 --2- 172.21.15.52:0/4067176507 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f106c0ce190 0x7f106c0ce600 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f106400d150 tx=0x7f106401ec30 comp rx=0 tx=0).stop 2024-05-04T12:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 -- 172.21.15.52:0/4067176507 shutdown_connections 2024-05-04T12:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 --2- 172.21.15.52:0/4067176507 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f106c0ce190 0x7f106c0ce600 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 --2- 172.21.15.52:0/4067176507 >> v2:172.21.15.111:3300/0 conn(0x7f106c06eba0 0x7f106c15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 -- 172.21.15.52:0/4067176507 >> 172.21.15.52:0/4067176507 conn(0x7f106c067730 msgr2=0x7f106c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 -- 172.21.15.52:0/4067176507 shutdown_connections 2024-05-04T12:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 -- 172.21.15.52:0/4067176507 wait complete. 2024-05-04T12:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 Processor -- start 2024-05-04T12:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 -- start start 2024-05-04T12:18:23.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f106c06cc40 0x7f106c156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f106c06eba0 0x7f106c1594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f106c0ce190 0x7f106c158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f106c0cecc0 con 0x7f106c0ce190 2024-05-04T12:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f106c0ceb40 con 0x7f106c06cc40 2024-05-04T12:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1072f40640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f106c0cee40 con 0x7f106c06eba0 2024-05-04T12:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f10714dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f106c0ce190 0x7f106c158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f10714dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f106c0ce190 0x7f106c158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54196/0 (socket says 172.21.15.52:54196) 2024-05-04T12:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f10714dd640 1 -- 172.21.15.52:0/1565491635 learned_addr learned my addr 172.21.15.52:0/1565491635 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f106bfff640 1 --2- 172.21.15.52:0/1565491635 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f106c06eba0 0x7f106c1594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f1070cdc640 1 --2- 172.21.15.52:0/1565491635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f106c06cc40 0x7f106c156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f10714dd640 1 -- 172.21.15.52:0/1565491635 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f106c06eba0 msgr2=0x7f106c1594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f10714dd640 1 --2- 172.21.15.52:0/1565491635 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f106c06eba0 0x7f106c1594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f10714dd640 1 -- 172.21.15.52:0/1565491635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f106c06cc40 msgr2=0x7f106c156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f10714dd640 1 --2- 172.21.15.52:0/1565491635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f106c06cc40 0x7f106c156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.638+0000 7f10714dd640 1 -- 172.21.15.52:0/1565491635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106c159a00 con 0x7f106c0ce190 2024-05-04T12:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.642+0000 7f10714dd640 1 --2- 172.21.15.52:0/1565491635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f106c0ce190 0x7f106c158620 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f10640b1040 tx=0x7f106401b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.642+0000 7f10697fa640 1 -- 172.21.15.52:0/1565491635 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064009800 con 0x7f106c0ce190 2024-05-04T12:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.642+0000 7f10697fa640 1 -- 172.21.15.52:0/1565491635 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10640c0070 con 0x7f106c0ce190 2024-05-04T12:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.642+0000 7f1072f40640 1 -- 172.21.15.52:0/1565491635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106c157580 con 0x7f106c0ce190 2024-05-04T12:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.642+0000 7f10697fa640 1 -- 172.21.15.52:0/1565491635 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064009c00 con 0x7f106c0ce190 2024-05-04T12:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.642+0000 7f1072f40640 1 -- 172.21.15.52:0/1565491635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f106c157b40 con 0x7f106c0ce190 2024-05-04T12:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.642+0000 7f1072f40640 1 -- 172.21.15.52:0/1565491635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f106c062e50 con 0x7f106c0ce190 2024-05-04T12:18:23.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.642+0000 7f10697fa640 1 -- 172.21.15.52:0/1565491635 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f106400b2e0 con 0x7f106c0ce190 2024-05-04T12:18:23.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.642+0000 7f10697fa640 1 --2- 172.21.15.52:0/1565491635 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f10580678c0 0x7f1058069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:23.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.642+0000 7f10697fa640 1 -- 172.21.15.52:0/1565491635 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1064122fe0 con 0x7f106c0ce190 2024-05-04T12:18:23.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.642+0000 7f1070cdc640 1 --2- 172.21.15.52:0/1565491635 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f10580678c0 0x7f1058069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:23.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.642+0000 7f1070cdc640 1 --2- 172.21.15.52:0/1565491635 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f10580678c0 0x7f1058069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f105c001650 tx=0x7f105c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:23.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.646+0000 7f10697fa640 1 -- 172.21.15.52:0/1565491635 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1064018820 con 0x7f106c0ce190 2024-05-04T12:18:23.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:23.850+0000 7f1072f40640 1 -- 172.21.15.52:0/1565491635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f106c0cc1f0 con 0x7f106c0ce190 2024-05-04T12:18:24.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:24.866+0000 7f10697fa640 1 -- 172.21.15.52:0/1565491635 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7f10640b73a0 con 0x7f106c0ce190 2024-05-04T12:18:24.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:24.870+0000 7f1072f40640 1 -- 172.21.15.52:0/1565491635 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f10580678c0 msgr2=0x7f1058069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:24.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:24.870+0000 7f1072f40640 1 --2- 172.21.15.52:0/1565491635 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f10580678c0 0x7f1058069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f105c001650 tx=0x7f105c009040 comp rx=0 tx=0).stop 2024-05-04T12:18:24.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:24.870+0000 7f1072f40640 1 -- 172.21.15.52:0/1565491635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f106c0ce190 msgr2=0x7f106c158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:24.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:24.870+0000 7f1072f40640 1 --2- 172.21.15.52:0/1565491635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f106c0ce190 0x7f106c158620 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f10640b1040 tx=0x7f106401b9d0 comp rx=0 tx=0).stop 2024-05-04T12:18:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:24.870+0000 7f1072f40640 1 -- 172.21.15.52:0/1565491635 shutdown_connections 2024-05-04T12:18:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:24.870+0000 7f1072f40640 1 --2- 172.21.15.52:0/1565491635 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f10580678c0 0x7f1058069d80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:24.870+0000 7f1072f40640 1 --2- 172.21.15.52:0/1565491635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f106c0ce190 0x7f106c158620 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:24.870+0000 7f1072f40640 1 --2- 172.21.15.52:0/1565491635 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f106c06eba0 0x7f106c1594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:24.870+0000 7f1072f40640 1 --2- 172.21.15.52:0/1565491635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f106c06cc40 0x7f106c156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:24.870+0000 7f1072f40640 1 -- 172.21.15.52:0/1565491635 >> 172.21.15.52:0/1565491635 conn(0x7f106c067730 msgr2=0x7f106c0cd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:24.870+0000 7f1072f40640 1 -- 172.21.15.52:0/1565491635 shutdown_connections 2024-05-04T12:18:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:24.870+0000 7f1072f40640 1 -- 172.21.15.52:0/1565491635 wait complete. 2024-05-04T12:18:24.896 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-04T12:18:24.896 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-04T12:18:25.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.022+0000 7fd1c1698640 1 Processor -- start 2024-05-04T12:18:25.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.022+0000 7fd1c1698640 1 -- start start 2024-05-04T12:18:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.022+0000 7fd1c1698640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd1bc15e180 0x7fd1bc0ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.022+0000 7fd1c1698640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd1bc162660 0x7fd1bc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.022+0000 7fd1c1698640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1bc0ce6a0 con 0x7fd1bc070c10 2024-05-04T12:18:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.022+0000 7fd1c1698640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fd1bc0cf220 con 0x7fd1bc15e180 2024-05-04T12:18:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.022+0000 7fd1c1698640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fd1bc0ce980 con 0x7fd1bc162660 2024-05-04T12:18:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.022+0000 7fd1baffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd1bc15e180 0x7fd1bc0ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.022+0000 7fd1baffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd1bc15e180 0x7fd1bc0ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54208/0 (socket says 172.21.15.52:54208) 2024-05-04T12:18:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.022+0000 7fd1baffd640 1 -- 172.21.15.52:0/2238238722 learned_addr learned my addr 172.21.15.52:0/2238238722 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:25.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1ba7fc640 1 --2- 172.21.15.52:0/2238238722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd1bc162660 0x7fd1bc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:25.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1baffd640 1 -- 172.21.15.52:0/2238238722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd1bc162660 msgr2=0x7fd1bc0706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:25.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1baffd640 1 --2- 172.21.15.52:0/2238238722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd1bc162660 0x7fd1bc0706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:25.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1baffd640 1 -- 172.21.15.52:0/2238238722 >> v1:172.21.15.111:6789/0 conn(0x7fd1bc070c10 legacy=0x7fd1bc06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:25.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1baffd640 1 -- 172.21.15.52:0/2238238722 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc0cec60 con 0x7fd1bc15e180 2024-05-04T12:18:25.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1baffd640 1 --2- 172.21.15.52:0/2238238722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1bc15e180 0x7fd1bc0ce0c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009e80 tx=0x7fd1ac01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3da7cf8cb26022 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:25.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1b9ffb640 1 -- 172.21.15.52:0/2238238722 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac0a56e0 con 0x7fd1bc15e180 2024-05-04T12:18:25.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1b9ffb640 1 -- 172.21.15.52:0/2238238722 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ac0a5880 con 0x7fd1bc15e180 2024-05-04T12:18:25.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1b9ffb640 1 -- 172.21.15.52:0/2238238722 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac0a5c50 con 0x7fd1bc15e180 2024-05-04T12:18:25.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1c1698640 1 -- 172.21.15.52:0/2238238722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1bc15e180 msgr2=0x7fd1bc0ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:25.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1c1698640 1 --2- 172.21.15.52:0/2238238722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1bc15e180 0x7fd1bc0ce0c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009e80 tx=0x7fd1ac01ca10 comp rx=0 tx=0).stop 2024-05-04T12:18:25.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1c1698640 1 -- 172.21.15.52:0/2238238722 shutdown_connections 2024-05-04T12:18:25.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1c1698640 1 --2- 172.21.15.52:0/2238238722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd1bc162660 0x7fd1bc0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:25.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1c1698640 1 --2- 172.21.15.52:0/2238238722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1bc15e180 0x7fd1bc0ce0c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:25.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1c1698640 1 -- 172.21.15.52:0/2238238722 >> 172.21.15.52:0/2238238722 conn(0x7fd1bc0d12c0 msgr2=0x7fd1bc0d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:25.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1c1698640 1 -- 172.21.15.52:0/2238238722 shutdown_connections 2024-05-04T12:18:25.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.026+0000 7fd1c1698640 1 -- 172.21.15.52:0/2238238722 wait complete. 2024-05-04T12:18:25.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1c1698640 1 Processor -- start 2024-05-04T12:18:25.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1c1698640 1 -- start start 2024-05-04T12:18:25.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1c1698640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd1bc070c10 0x7fd1bc148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:25.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1c1698640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1bc15e180 0x7fd1bc160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:25.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1c1698640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1bc162660 0x7fd1bc153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:25.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1c1698640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc16b170 con 0x7fd1bc162660 2024-05-04T12:18:25.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1c1698640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc16aff0 con 0x7fd1bc15e180 2024-05-04T12:18:25.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1c1698640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fd1bc16b2f0 con 0x7fd1bc070c10 2024-05-04T12:18:25.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1bc162660 0x7fd1bc153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:25.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1bc162660 0x7fd1bc153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54214/0 (socket says 172.21.15.52:54214) 2024-05-04T12:18:25.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1bb7fe640 1 -- 172.21.15.52:0/852491214 learned_addr learned my addr 172.21.15.52:0/852491214 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:25.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1ba7fc640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1bc15e180 0x7fd1bc160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:25.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1baffd640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd1bc070c10 0x7fd1bc148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:25.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1bb7fe640 1 -- 172.21.15.52:0/852491214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd1bc070c10 msgr2=0x7fd1bc148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:25.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1bb7fe640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd1bc070c10 0x7fd1bc148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:25.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1bb7fe640 1 -- 172.21.15.52:0/852491214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1bc15e180 msgr2=0x7fd1bc160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:25.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1bb7fe640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1bc15e180 0x7fd1bc160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:25.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1bb7fe640 1 -- 172.21.15.52:0/852491214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc149350 con 0x7fd1bc162660 2024-05-04T12:18:25.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1baffd640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd1bc070c10 0x7fd1bc148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:25.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1bb7fe640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1bc162660 0x7fd1bc153eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400c430 tx=0x7fd1b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:25.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1b37fe640 1 -- 172.21.15.52:0/852491214 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4005220 con 0x7fd1bc162660 2024-05-04T12:18:25.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1b37fe640 1 -- 172.21.15.52:0/852491214 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b40053c0 con 0x7fd1bc162660 2024-05-04T12:18:25.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1b37fe640 1 -- 172.21.15.52:0/852491214 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4007270 con 0x7fd1bc162660 2024-05-04T12:18:25.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1c1698640 1 -- 172.21.15.52:0/852491214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1bc15ea00 con 0x7fd1bc162660 2024-05-04T12:18:25.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1c1698640 1 -- 172.21.15.52:0/852491214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1bc15efc0 con 0x7fd1bc162660 2024-05-04T12:18:25.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.030+0000 7fd1c1698640 1 -- 172.21.15.52:0/852491214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1bc062e50 con 0x7fd1bc162660 2024-05-04T12:18:25.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.034+0000 7fd1b37fe640 1 -- 172.21.15.52:0/852491214 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd1b4019020 con 0x7fd1bc162660 2024-05-04T12:18:25.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.034+0000 7fd1b37fe640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd19c0678c0 0x7fd19c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:25.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.034+0000 7fd1b37fe640 1 -- 172.21.15.52:0/852491214 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd1b4085060 con 0x7fd1bc162660 2024-05-04T12:18:25.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.034+0000 7fd1baffd640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd19c0678c0 0x7fd19c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:25.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.034+0000 7fd1baffd640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd19c0678c0 0x7fd19c069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0ae000 tx=0x7fd1ac0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:25.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.038+0000 7fd1b37fe640 1 -- 172.21.15.52:0/852491214 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd1b4055b90 con 0x7fd1bc162660 2024-05-04T12:18:25.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.234+0000 7fd1c1698640 1 -- 172.21.15.52:0/852491214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd1bc0008e0 con 0x7fd1bc162660 2024-05-04T12:18:25.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.866+0000 7fd1b37fe640 1 -- 172.21.15.52:0/852491214 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7fd1b4055490 con 0x7fd1bc162660 2024-05-04T12:18:25.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.870+0000 7fd1c1698640 1 -- 172.21.15.52:0/852491214 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd19c0678c0 msgr2=0x7fd19c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:25.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.870+0000 7fd1c1698640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd19c0678c0 0x7fd19c069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0ae000 tx=0x7fd1ac0a7070 comp rx=0 tx=0).stop 2024-05-04T12:18:25.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.870+0000 7fd1c1698640 1 -- 172.21.15.52:0/852491214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1bc162660 msgr2=0x7fd1bc153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:25.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.870+0000 7fd1c1698640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1bc162660 0x7fd1bc153eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400c430 tx=0x7fd1b400c8f0 comp rx=0 tx=0).stop 2024-05-04T12:18:25.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.870+0000 7fd1c1698640 1 -- 172.21.15.52:0/852491214 shutdown_connections 2024-05-04T12:18:25.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.870+0000 7fd1c1698640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd19c0678c0 0x7fd19c069d80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:25.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.870+0000 7fd1c1698640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1bc162660 0x7fd1bc153eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:25.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.870+0000 7fd1c1698640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1bc15e180 0x7fd1bc160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:25.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.870+0000 7fd1c1698640 1 --2- 172.21.15.52:0/852491214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd1bc070c10 0x7fd1bc148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:25.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.870+0000 7fd1c1698640 1 -- 172.21.15.52:0/852491214 >> 172.21.15.52:0/852491214 conn(0x7fd1bc0d12c0 msgr2=0x7fd1bc0d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:25.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.870+0000 7fd1c1698640 1 -- 172.21.15.52:0/852491214 shutdown_connections 2024-05-04T12:18:25.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:25.870+0000 7fd1c1698640 1 -- 172.21.15.52:0/852491214 wait complete. 2024-05-04T12:18:25.899 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-04T12:18:25.899 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T12:18:26.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad1a818640 1 Processor -- start 2024-05-04T12:18:26.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad1a818640 1 -- start start 2024-05-04T12:18:26.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad1a818640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad141441b0 0x7fad141627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad1a818640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fad140b6bf0 0x7fad14141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad1a818640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad14162cf0 con 0x7fad14144590 2024-05-04T12:18:26.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad1a818640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fad1415f6d0 con 0x7fad140b6bf0 2024-05-04T12:18:26.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad1a818640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fad1415eac0 con 0x7fad141441b0 2024-05-04T12:18:26.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad18db5640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fad140b6bf0 0x7fad14141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad137fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fad14144590 0x7fad140b64e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58934/0 (socket says 172.21.15.52:58934) 2024-05-04T12:18:26.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad137fe640 1 -- 172.21.15.52:0/3289419221 learned_addr learned my addr 172.21.15.52:0/3289419221 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:26.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad13fff640 1 --2- 172.21.15.52:0/3289419221 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad141441b0 0x7fad141627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad18db5640 1 -- 172.21.15.52:0/3289419221 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad141441b0 msgr2=0x7fad141627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:26.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad18db5640 1 --2- 172.21.15.52:0/3289419221 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad141441b0 0x7fad141627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad18db5640 1 -- 172.21.15.52:0/3289419221 >> v1:172.21.15.52:6789/0 conn(0x7fad14144590 legacy=0x7fad140b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad12ffd640 1 -- 172.21.15.52:0/3289419221 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3301005706 0 0) 0x7fad08001120 con 0x7fad14144590 2024-05-04T12:18:26.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.026+0000 7fad18db5640 1 -- 172.21.15.52:0/3289419221 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1415eda0 con 0x7fad140b6bf0 2024-05-04T12:18:26.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad18db5640 1 --2- 172.21.15.52:0/3289419221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad140b6bf0 0x7fad14141ab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009d90 tx=0x7fad0c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5531cc131bd4147a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:26.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad12ffd640 1 -- 172.21.15.52:0/3289419221 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0b2070 con 0x7fad140b6bf0 2024-05-04T12:18:26.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad12ffd640 1 -- 172.21.15.52:0/3289419221 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0c0a5a00 con 0x7fad140b6bf0 2024-05-04T12:18:26.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad12ffd640 1 -- 172.21.15.52:0/3289419221 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0a5dd0 con 0x7fad140b6bf0 2024-05-04T12:18:26.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 -- 172.21.15.52:0/3289419221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad140b6bf0 msgr2=0x7fad14141ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 --2- 172.21.15.52:0/3289419221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad140b6bf0 0x7fad14141ab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009d90 tx=0x7fad0c01c940 comp rx=0 tx=0).stop 2024-05-04T12:18:26.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 -- 172.21.15.52:0/3289419221 shutdown_connections 2024-05-04T12:18:26.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 --2- 172.21.15.52:0/3289419221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad140b6bf0 0x7fad14141ab0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 --2- 172.21.15.52:0/3289419221 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad141441b0 0x7fad141627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 -- 172.21.15.52:0/3289419221 >> 172.21.15.52:0/3289419221 conn(0x7fad140b52d0 msgr2=0x7fad140b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:26.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 -- 172.21.15.52:0/3289419221 shutdown_connections 2024-05-04T12:18:26.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 -- 172.21.15.52:0/3289419221 wait complete. 2024-05-04T12:18:26.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 Processor -- start 2024-05-04T12:18:26.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 -- start start 2024-05-04T12:18:26.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad140b6bf0 0x7fad1413f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad141441b0 0x7fad14157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad14144590 0x7fad141f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fad14143c10 con 0x7fad14144590 2024-05-04T12:18:26.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fad14143a90 con 0x7fad141441b0 2024-05-04T12:18:26.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad1a818640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fad14142800 con 0x7fad140b6bf0 2024-05-04T12:18:26.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad18db5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad14144590 0x7fad141f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad18db5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad14144590 0x7fad141f7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54224/0 (socket says 172.21.15.52:54224) 2024-05-04T12:18:26.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.030+0000 7fad18db5640 1 -- 172.21.15.52:0/2638301892 learned_addr learned my addr 172.21.15.52:0/2638301892 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:26.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad13fff640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad140b6bf0 0x7fad1413f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad137fe640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad141441b0 0x7fad14157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad18db5640 1 -- 172.21.15.52:0/2638301892 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad140b6bf0 msgr2=0x7fad1413f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad18db5640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad140b6bf0 0x7fad1413f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad18db5640 1 -- 172.21.15.52:0/2638301892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad141441b0 msgr2=0x7fad14157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad18db5640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad141441b0 0x7fad14157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad18db5640 1 -- 172.21.15.52:0/2638301892 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1413fce0 con 0x7fad14144590 2024-05-04T12:18:26.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad13fff640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad140b6bf0 0x7fad1413f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:26.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad18db5640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad14144590 0x7fad141f7e50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad0c01aff0 tx=0x7fad0c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:26.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad10ff9640 1 -- 172.21.15.52:0/2638301892 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0b2050 con 0x7fad14144590 2024-05-04T12:18:26.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad10ff9640 1 -- 172.21.15.52:0/2638301892 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0c0be070 con 0x7fad14144590 2024-05-04T12:18:26.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad10ff9640 1 -- 172.21.15.52:0/2638301892 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0adb50 con 0x7fad14144590 2024-05-04T12:18:26.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad1a818640 1 -- 172.21.15.52:0/2638301892 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad141586a0 con 0x7fad14144590 2024-05-04T12:18:26.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad1a818640 1 -- 172.21.15.52:0/2638301892 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1413efa0 con 0x7fad14144590 2024-05-04T12:18:26.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad1a818640 1 -- 172.21.15.52:0/2638301892 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad14062e50 con 0x7fad14144590 2024-05-04T12:18:26.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad10ff9640 1 -- 172.21.15.52:0/2638301892 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fad0c0adcf0 con 0x7fad14144590 2024-05-04T12:18:26.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad10ff9640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fad000678c0 0x7fad00069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad10ff9640 1 -- 172.21.15.52:0/2638301892 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fad0c1211d0 con 0x7fad14144590 2024-05-04T12:18:26.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad13fff640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fad000678c0 0x7fad00069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.034+0000 7fad13fff640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fad000678c0 0x7fad00069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fad04009840 tx=0x7fad04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:26.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.038+0000 7fad10ff9640 1 -- 172.21.15.52:0/2638301892 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fad0c016560 con 0x7fad14144590 2024-05-04T12:18:26.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.234+0000 7fad1a818640 1 -- 172.21.15.52:0/2638301892 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad14141550 con 0x7fad14144590 2024-05-04T12:18:26.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.234+0000 7fad10ff9640 1 -- 172.21.15.52:0/2638301892 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7fad0c0ab340 con 0x7fad14144590 2024-05-04T12:18:26.239 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:26.239 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6836/482177111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":482177111},{"type":"v1","addr":"172.21.15.52:6836","nonce":482177111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/163453709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6835","nonce":163453709},{"type":"v1","addr":"172.21.15.52:6837","nonce":163453709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3596668218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3596668218},{"type":"v1","addr":"172.21.15.111:6833","nonce":3596668218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-04T12:18:22.189525+0000","modified":"2024-05-04T12:18:25.240402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/1656435333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1656435333},{"type":"v1","addr":"172.21.15.111:6835","nonce":1656435333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-05-04T12:18:26.239 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 7 2024-05-04T12:18:26.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.238+0000 7fad1a818640 1 -- 172.21.15.52:0/2638301892 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fad000678c0 msgr2=0x7fad00069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.238+0000 7fad1a818640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fad000678c0 0x7fad00069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fad04009840 tx=0x7fad04008040 comp rx=0 tx=0).stop 2024-05-04T12:18:26.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.238+0000 7fad1a818640 1 -- 172.21.15.52:0/2638301892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad14144590 msgr2=0x7fad141f7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.238+0000 7fad1a818640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad14144590 0x7fad141f7e50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad0c01aff0 tx=0x7fad0c0196e0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.238+0000 7fad1a818640 1 -- 172.21.15.52:0/2638301892 shutdown_connections 2024-05-04T12:18:26.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.238+0000 7fad1a818640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fad000678c0 0x7fad00069d80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.238+0000 7fad1a818640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad14144590 0x7fad141f7e50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.238+0000 7fad1a818640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad141441b0 0x7fad14157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.238+0000 7fad1a818640 1 --2- 172.21.15.52:0/2638301892 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad140b6bf0 0x7fad1413f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.238+0000 7fad1a818640 1 -- 172.21.15.52:0/2638301892 >> 172.21.15.52:0/2638301892 conn(0x7fad140b52d0 msgr2=0x7fad14161000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:26.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.238+0000 7fad1a818640 1 -- 172.21.15.52:0/2638301892 shutdown_connections 2024-05-04T12:18:26.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.238+0000 7fad1a818640 1 -- 172.21.15.52:0/2638301892 wait complete. 2024-05-04T12:18:26.257 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:26.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9f1502640 1 Processor -- start 2024-05-04T12:18:26.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9f1502640 1 -- start start 2024-05-04T12:18:26.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9f1502640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd9ec1441b0 0x7fd9ec1627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9f1502640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd9ec144590 0x7fd9ec0b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9f1502640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9ec162cf0 con 0x7fd9ec0b6a20 2024-05-04T12:18:26.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9f1502640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fd9ec0b6e80 con 0x7fd9ec144590 2024-05-04T12:18:26.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9f1502640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fd9ec15f6d0 con 0x7fd9ec1441b0 2024-05-04T12:18:26.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9eb7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fd9ec0b6a20 0x7fd9ec071680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51472/0 (socket says 172.21.15.52:51472) 2024-05-04T12:18:26.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9ea7fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd9ec144590 0x7fd9ec0b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9eb7fe640 1 -- 172.21.15.52:0/184115381 learned_addr learned my addr 172.21.15.52:0/184115381 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:26.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9eaffd640 1 --2- 172.21.15.52:0/184115381 >> v2:172.21.15.111:3300/0 conn(0x7fd9ec1441b0 0x7fd9ec1627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9e9ffb640 1 -- 172.21.15.52:0/184115381 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 275761329 0 0) 0x7fd9e4001120 con 0x7fd9ec0b6a20 2024-05-04T12:18:26.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9e9ffb640 1 -- 172.21.15.52:0/184115381 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9d8003530 con 0x7fd9ec0b6a20 2024-05-04T12:18:26.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9ea7fc640 1 -- 172.21.15.52:0/184115381 >> v2:172.21.15.111:3300/0 conn(0x7fd9ec1441b0 msgr2=0x7fd9ec1627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:26.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9ea7fc640 1 --2- 172.21.15.52:0/184115381 >> v2:172.21.15.111:3300/0 conn(0x7fd9ec1441b0 0x7fd9ec1627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9ea7fc640 1 -- 172.21.15.52:0/184115381 >> v1:172.21.15.52:6789/0 conn(0x7fd9ec0b6a20 legacy=0x7fd9ec071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9ea7fc640 1 -- 172.21.15.52:0/184115381 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ec15eac0 con 0x7fd9ec144590 2024-05-04T12:18:26.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.382+0000 7fd9eaffd640 1 --2- 172.21.15.52:0/184115381 >> v2:172.21.15.111:3300/0 conn(0x7fd9ec1441b0 0x7fd9ec1627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:26.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9ea7fc640 1 --2- 172.21.15.52:0/184115381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec144590 0x7fd9ec0b64e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec070d70 tx=0x7fd9dc01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3d208fcf578882 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:26.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9e9ffb640 1 -- 172.21.15.52:0/184115381 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc0b2070 con 0x7fd9ec144590 2024-05-04T12:18:26.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9e9ffb640 1 -- 172.21.15.52:0/184115381 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9dc0a59c0 con 0x7fd9ec144590 2024-05-04T12:18:26.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9e9ffb640 1 -- 172.21.15.52:0/184115381 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc0a5d90 con 0x7fd9ec144590 2024-05-04T12:18:26.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 -- 172.21.15.52:0/184115381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec144590 msgr2=0x7fd9ec0b64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 --2- 172.21.15.52:0/184115381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec144590 0x7fd9ec0b64e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec070d70 tx=0x7fd9dc01c900 comp rx=0 tx=0).stop 2024-05-04T12:18:26.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 -- 172.21.15.52:0/184115381 shutdown_connections 2024-05-04T12:18:26.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 --2- 172.21.15.52:0/184115381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec144590 0x7fd9ec0b64e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 --2- 172.21.15.52:0/184115381 >> v2:172.21.15.111:3300/0 conn(0x7fd9ec1441b0 0x7fd9ec1627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 -- 172.21.15.52:0/184115381 >> 172.21.15.52:0/184115381 conn(0x7fd9ec0b52d0 msgr2=0x7fd9ec0b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:26.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 -- 172.21.15.52:0/184115381 shutdown_connections 2024-05-04T12:18:26.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 -- 172.21.15.52:0/184115381 wait complete. 2024-05-04T12:18:26.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 Processor -- start 2024-05-04T12:18:26.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 -- start start 2024-05-04T12:18:26.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9ec0b6a20 0x7fd9ec13ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9ec1441b0 0x7fd9ec157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec144590 0x7fd9ec1580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec143a90 con 0x7fd9ec144590 2024-05-04T12:18:26.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec15e760 con 0x7fd9ec0b6a20 2024-05-04T12:18:26.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9f1502640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ec143c10 con 0x7fd9ec1441b0 2024-05-04T12:18:26.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9eb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec144590 0x7fd9ec1580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9eb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec144590 0x7fd9ec1580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53276/0 (socket says 172.21.15.52:53276) 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9eb7fe640 1 -- 172.21.15.52:0/4206270025 learned_addr learned my addr 172.21.15.52:0/4206270025 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9ea7fc640 1 --2- 172.21.15.52:0/4206270025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9ec1441b0 0x7fd9ec157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9eaffd640 1 --2- 172.21.15.52:0/4206270025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9ec0b6a20 0x7fd9ec13ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9eb7fe640 1 -- 172.21.15.52:0/4206270025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9ec1441b0 msgr2=0x7fd9ec157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9eb7fe640 1 --2- 172.21.15.52:0/4206270025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9ec1441b0 0x7fd9ec157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9eb7fe640 1 -- 172.21.15.52:0/4206270025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9ec0b6a20 msgr2=0x7fd9ec13ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9eb7fe640 1 --2- 172.21.15.52:0/4206270025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9ec0b6a20 0x7fd9ec13ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9eb7fe640 1 -- 172.21.15.52:0/4206270025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ec13f4a0 con 0x7fd9ec144590 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9eb7fe640 1 --2- 172.21.15.52:0/4206270025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec144590 0x7fd9ec1580b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9e400c730 tx=0x7fd9e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9e37fe640 1 -- 172.21.15.52:0/4206270025 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4017070 con 0x7fd9ec144590 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9e37fe640 1 -- 172.21.15.52:0/4206270025 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e40058c0 con 0x7fd9ec144590 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.386+0000 7fd9e37fe640 1 -- 172.21.15.52:0/4206270025 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4005cd0 con 0x7fd9ec144590 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.390+0000 7fd9f1502640 1 -- 172.21.15.52:0/4206270025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ec1388f0 con 0x7fd9ec144590 2024-05-04T12:18:26.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.390+0000 7fd9f1502640 1 -- 172.21.15.52:0/4206270025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ec138e90 con 0x7fd9ec144590 2024-05-04T12:18:26.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.390+0000 7fd9f1502640 1 -- 172.21.15.52:0/4206270025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9ec062e50 con 0x7fd9ec144590 2024-05-04T12:18:26.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.390+0000 7fd9e37fe640 1 -- 172.21.15.52:0/4206270025 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd9e401a070 con 0x7fd9ec144590 2024-05-04T12:18:26.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.390+0000 7fd9e37fe640 1 --2- 172.21.15.52:0/4206270025 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd9cc0678c0 0x7fd9cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.390+0000 7fd9e37fe640 1 -- 172.21.15.52:0/4206270025 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd9e40850d0 con 0x7fd9ec144590 2024-05-04T12:18:26.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.390+0000 7fd9eaffd640 1 --2- 172.21.15.52:0/4206270025 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd9cc0678c0 0x7fd9cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.390+0000 7fd9eaffd640 1 --2- 172.21.15.52:0/4206270025 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd9cc0678c0 0x7fd9cc069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400d8b0 tx=0x7fd9d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:26.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.394+0000 7fd9e37fe640 1 -- 172.21.15.52:0/4206270025 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd9ec162cf0 con 0x7fd9ec144590 2024-05-04T12:18:26.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.558+0000 7fd9f1502640 1 -- 172.21.15.52:0/4206270025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9ec141550 con 0x7fd9ec144590 2024-05-04T12:18:26.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.558+0000 7fd9e37fe640 1 -- 172.21.15.52:0/4206270025 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fd9e4055b80 con 0x7fd9ec144590 2024-05-04T12:18:26.562 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:26.563 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:26.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.562+0000 7fd9f1502640 1 -- 172.21.15.52:0/4206270025 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd9cc0678c0 msgr2=0x7fd9cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.562+0000 7fd9f1502640 1 --2- 172.21.15.52:0/4206270025 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd9cc0678c0 0x7fd9cc069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400d8b0 tx=0x7fd9d4008040 comp rx=0 tx=0).stop 2024-05-04T12:18:26.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.562+0000 7fd9f1502640 1 -- 172.21.15.52:0/4206270025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec144590 msgr2=0x7fd9ec1580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.562+0000 7fd9f1502640 1 --2- 172.21.15.52:0/4206270025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec144590 0x7fd9ec1580b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9e400c730 tx=0x7fd9e400cbf0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.562+0000 7fd9f1502640 1 -- 172.21.15.52:0/4206270025 shutdown_connections 2024-05-04T12:18:26.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.562+0000 7fd9f1502640 1 --2- 172.21.15.52:0/4206270025 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd9cc0678c0 0x7fd9cc069d80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.562+0000 7fd9f1502640 1 --2- 172.21.15.52:0/4206270025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec144590 0x7fd9ec1580b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.562+0000 7fd9f1502640 1 --2- 172.21.15.52:0/4206270025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9ec1441b0 0x7fd9ec157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.562+0000 7fd9f1502640 1 --2- 172.21.15.52:0/4206270025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9ec0b6a20 0x7fd9ec13ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.562+0000 7fd9f1502640 1 -- 172.21.15.52:0/4206270025 >> 172.21.15.52:0/4206270025 conn(0x7fd9ec0b52d0 msgr2=0x7fd9ec1602f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:26.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.562+0000 7fd9f1502640 1 -- 172.21.15.52:0/4206270025 shutdown_connections 2024-05-04T12:18:26.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.562+0000 7fd9f1502640 1 -- 172.21.15.52:0/4206270025 wait complete. 2024-05-04T12:18:26.581 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-04T12:18:26.581 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-04T12:18:26.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.710+0000 7f87c1741640 1 Processor -- start 2024-05-04T12:18:26.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.710+0000 7f87c1741640 1 -- start start 2024-05-04T12:18:26.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.710+0000 7f87c1741640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f87bc147940 0x7f87bc1621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.710+0000 7f87c1741640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87bc1627d0 con 0x7f87bc15f6a0 2024-05-04T12:18:26.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.710+0000 7f87c1741640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87bc162ab0 con 0x7f87bc0d05a0 2024-05-04T12:18:26.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.710+0000 7f87c1741640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f87bc15fef0 con 0x7f87bc147940 2024-05-04T12:18:26.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87b3fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f87bc15f6a0 0x7f87bc0cfe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51474/0 (socket says 172.21.15.52:51474) 2024-05-04T12:18:26.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87b3fff640 1 -- 172.21.15.52:0/3114325267 learned_addr learned my addr 172.21.15.52:0/3114325267 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:26.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87baffd640 1 --2- 172.21.15.52:0/3114325267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f87bc147940 0x7f87bc1621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 -- 172.21.15.52:0/3114325267 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 685649389 0 0) 0x7f87bc1627d0 con 0x7f87bc15f6a0 2024-05-04T12:18:26.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 -- 172.21.15.52:0/3114325267 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87a8003530 con 0x7f87bc15f6a0 2024-05-04T12:18:26.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 -- 172.21.15.52:0/3114325267 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3475736795 0 0) 0x7f87a8003530 con 0x7f87bc15f6a0 2024-05-04T12:18:26.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 -- 172.21.15.52:0/3114325267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f87bc147940 msgr2=0x7f87bc1621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:26.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 --2- 172.21.15.52:0/3114325267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f87bc147940 0x7f87bc1621f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 -- 172.21.15.52:0/3114325267 >> v1:172.21.15.111:6789/0 conn(0x7f87bc0d05a0 legacy=0x7f87bc160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 -- 172.21.15.52:0/3114325267 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87bc162d90 con 0x7f87bc15f6a0 2024-05-04T12:18:26.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 -- 172.21.15.52:0/3114325267 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f87a4001680 con 0x7f87bc15f6a0 2024-05-04T12:18:26.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 -- 172.21.15.52:0/3114325267 >> v1:172.21.15.52:6789/0 conn(0x7f87bc15f6a0 legacy=0x7f87bc0cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 --2- 172.21.15.52:0/3114325267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87a80039f0 0x7f87a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 --2- 172.21.15.52:0/3114325267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87a8004810 0x7f87a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 --2- 172.21.15.52:0/3114325267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f87bc147940 0x7f87a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 -- 172.21.15.52:0/3114325267 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc15fef0 con 0x7f87a8004810 2024-05-04T12:18:26.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 -- 172.21.15.52:0/3114325267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f87a800ba80 con 0x7f87a80039f0 2024-05-04T12:18:26.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 -- 172.21.15.52:0/3114325267 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f87a800bc80 con 0x7f87bc147940 2024-05-04T12:18:26.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87b3fff640 1 --2- 172.21.15.52:0/3114325267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87a8004810 0x7f87a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87baffd640 1 --2- 172.21.15.52:0/3114325267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87a80039f0 0x7f87a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87bb7fe640 1 --2- 172.21.15.52:0/3114325267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f87bc147940 0x7f87a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87b3fff640 1 -- 172.21.15.52:0/3114325267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f87bc147940 msgr2=0x7f87a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:26.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87b3fff640 1 --2- 172.21.15.52:0/3114325267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f87bc147940 0x7f87a80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87b3fff640 1 -- 172.21.15.52:0/3114325267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87a80039f0 msgr2=0x7f87a8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:26.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87b3fff640 1 --2- 172.21.15.52:0/3114325267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87a80039f0 0x7f87a8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87b3fff640 1 -- 172.21.15.52:0/3114325267 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a8003530 con 0x7f87a8004810 2024-05-04T12:18:26.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87b3fff640 1 --2- 172.21.15.52:0/3114325267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87a8004810 0x7f87a8004c90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c190 tx=0x7f87a401f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c265f7655f158a16 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:26.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 -- 172.21.15.52:0/3114325267 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4007d00 con 0x7f87a8004810 2024-05-04T12:18:26.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 -- 172.21.15.52:0/3114325267 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a4005b00 con 0x7f87a8004810 2024-05-04T12:18:26.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.714+0000 7f87ba7fc640 1 -- 172.21.15.52:0/3114325267 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4007070 con 0x7f87a8004810 2024-05-04T12:18:26.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 -- 172.21.15.52:0/3114325267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87a8004810 msgr2=0x7f87a8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 --2- 172.21.15.52:0/3114325267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87a8004810 0x7f87a8004c90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c190 tx=0x7f87a401f370 comp rx=0 tx=0).stop 2024-05-04T12:18:26.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87bb7fe640 1 -- 172.21.15.52:0/3114325267 reap_dead start 2024-05-04T12:18:26.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 -- 172.21.15.52:0/3114325267 shutdown_connections 2024-05-04T12:18:26.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 -- 172.21.15.52:0/3114325267 >> 172.21.15.52:0/3114325267 conn(0x7f87bc0d0f80 msgr2=0x7f87bc0d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:26.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 -- 172.21.15.52:0/3114325267 shutdown_connections 2024-05-04T12:18:26.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 -- 172.21.15.52:0/3114325267 wait complete. 2024-05-04T12:18:26.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 Processor -- start 2024-05-04T12:18:26.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 -- start start 2024-05-04T12:18:26.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f87bc1e9d30 0x7f87bc1ea110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87bc1e9280 0x7f87bc1400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87bc140610 0x7f87bc0aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc1621f0 con 0x7f87bc140610 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc15fb00 con 0x7f87bc1e9280 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f87bc162370 con 0x7f87bc1e9d30 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87bc140610 0x7f87bc0aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87b3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87bc1e9280 0x7f87bc1400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87bc140610 0x7f87bc0aa3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53292/0 (socket says 172.21.15.52:53292) 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87bb7fe640 1 -- 172.21.15.52:0/1127110915 learned_addr learned my addr 172.21.15.52:0/1127110915 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87baffd640 1 --2- 172.21.15.52:0/1127110915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f87bc1e9d30 0x7f87bc1ea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87bb7fe640 1 -- 172.21.15.52:0/1127110915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f87bc1e9d30 msgr2=0x7f87bc1ea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87bb7fe640 1 --2- 172.21.15.52:0/1127110915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f87bc1e9d30 0x7f87bc1ea110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87bb7fe640 1 -- 172.21.15.52:0/1127110915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87bc1e9280 msgr2=0x7f87bc1400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87bb7fe640 1 --2- 172.21.15.52:0/1127110915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87bc1e9280 0x7f87bc1400d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87bb7fe640 1 -- 172.21.15.52:0/1127110915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87bc1ea650 con 0x7f87bc140610 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87bb7fe640 1 --2- 172.21.15.52:0/1127110915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87bc140610 0x7f87bc0aa3e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f87bc15fe60 tx=0x7f87b400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:26.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87b37fe640 1 -- 172.21.15.52:0/1127110915 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4003980 con 0x7f87bc140610 2024-05-04T12:18:26.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87b37fe640 1 -- 172.21.15.52:0/1127110915 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87b4018070 con 0x7f87bc140610 2024-05-04T12:18:26.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 -- 172.21.15.52:0/1127110915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87bc1e81e0 con 0x7f87bc140610 2024-05-04T12:18:26.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87b37fe640 1 -- 172.21.15.52:0/1127110915 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4003d60 con 0x7f87bc140610 2024-05-04T12:18:26.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.718+0000 7f87c1741640 1 -- 172.21.15.52:0/1127110915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87bc1e87a0 con 0x7f87bc140610 2024-05-04T12:18:26.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.722+0000 7f87c1741640 1 -- 172.21.15.52:0/1127110915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87bc062e50 con 0x7f87bc140610 2024-05-04T12:18:26.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.722+0000 7f87b37fe640 1 -- 172.21.15.52:0/1127110915 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f87b40050d0 con 0x7f87bc140610 2024-05-04T12:18:26.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.722+0000 7f87b37fe640 1 --2- 172.21.15.52:0/1127110915 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f879c0678f0 0x7f879c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:26.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.722+0000 7f87b37fe640 1 -- 172.21.15.52:0/1127110915 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f87b4085890 con 0x7f87bc140610 2024-05-04T12:18:26.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.722+0000 7f87baffd640 1 --2- 172.21.15.52:0/1127110915 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f879c0678f0 0x7f879c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:26.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.722+0000 7f87baffd640 1 --2- 172.21.15.52:0/1127110915 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f879c0678f0 0x7f879c069db0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f87bc1f7e70 tx=0x7f87ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:26.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.726+0000 7f87b37fe640 1 -- 172.21.15.52:0/1127110915 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f87b4055d80 con 0x7f87bc140610 2024-05-04T12:18:26.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.886+0000 7f87c1741640 1 -- 172.21.15.52:0/1127110915 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f87bc075320 con 0x7f879c0678f0 2024-05-04T12:18:26.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.894+0000 7f87b37fe640 1 -- 172.21.15.52:0/1127110915 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7f87bc075320 con 0x7f879c0678f0 2024-05-04T12:18:26.899 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:26.901 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-04T12:18:26.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.898+0000 7f87c1741640 1 -- 172.21.15.52:0/1127110915 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f879c0678f0 msgr2=0x7f879c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.898+0000 7f87c1741640 1 --2- 172.21.15.52:0/1127110915 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f879c0678f0 0x7f879c069db0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f87bc1f7e70 tx=0x7f87ac008040 comp rx=0 tx=0).stop 2024-05-04T12:18:26.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.898+0000 7f87c1741640 1 -- 172.21.15.52:0/1127110915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87bc140610 msgr2=0x7f87bc0aa3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:26.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.898+0000 7f87c1741640 1 --2- 172.21.15.52:0/1127110915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87bc140610 0x7f87bc0aa3e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f87bc15fe60 tx=0x7f87b400ec10 comp rx=0 tx=0).stop 2024-05-04T12:18:26.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.898+0000 7f87c1741640 1 -- 172.21.15.52:0/1127110915 shutdown_connections 2024-05-04T12:18:26.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.898+0000 7f87c1741640 1 --2- 172.21.15.52:0/1127110915 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f879c0678f0 0x7f879c069db0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.898+0000 7f87c1741640 1 --2- 172.21.15.52:0/1127110915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87bc140610 0x7f87bc0aa3e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.898+0000 7f87c1741640 1 --2- 172.21.15.52:0/1127110915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87bc1e9280 0x7f87bc1400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.898+0000 7f87c1741640 1 --2- 172.21.15.52:0/1127110915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f87bc1e9d30 0x7f87bc1ea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:26.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.898+0000 7f87c1741640 1 -- 172.21.15.52:0/1127110915 >> 172.21.15.52:0/1127110915 conn(0x7f87bc0d0f80 msgr2=0x7f87bc070600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:26.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.898+0000 7f87c1741640 1 -- 172.21.15.52:0/1127110915 shutdown_connections 2024-05-04T12:18:26.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:26.898+0000 7f87c1741640 1 -- 172.21.15.52:0/1127110915 wait complete. 2024-05-04T12:18:26.918 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-04T12:18:26.741229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235708,"kb_used_data":2884,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739012,"statfs":{"total":773094113280,"available":772852748288,"internally_reserved":0,"allocated":2953216,"data_stored":1771496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.936852"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298061+0000","last_change":"2024-05-04T12:18:22.859306+0000","last_active":"2024-05-04T12:18:24.298061+0000","last_peered":"2024-05-04T12:18:24.298061+0000","last_clean":"2024-05-04T12:18:24.298061+0000","last_became_active":"2024-05-04T12:18:22.859107+0000","last_became_peered":"2024-05-04T12:18:22.859107+0000","last_unstale":"2024-05-04T12:18:24.298061+0000","last_undegraded":"2024-05-04T12:18:24.298061+0000","last_fullsized":"2024-05-04T12:18:24.298061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:45:17.602528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.832155+0000","last_change":"2024-05-04T12:18:22.848885+0000","last_active":"2024-05-04T12:18:23.832155+0000","last_peered":"2024-05-04T12:18:23.832155+0000","last_clean":"2024-05-04T12:18:23.832155+0000","last_became_active":"2024-05-04T12:18:22.848320+0000","last_became_peered":"2024-05-04T12:18:22.848320+0000","last_unstale":"2024-05-04T12:18:23.832155+0000","last_undegraded":"2024-05-04T12:18:23.832155+0000","last_fullsized":"2024-05-04T12:18:23.832155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247882+0000","last_change":"2024-05-04T12:18:22.861471+0000","last_active":"2024-05-04T12:18:24.247882+0000","last_peered":"2024-05-04T12:18:24.247882+0000","last_clean":"2024-05-04T12:18:24.247882+0000","last_became_active":"2024-05-04T12:18:22.860827+0000","last_became_peered":"2024-05-04T12:18:22.860827+0000","last_unstale":"2024-05-04T12:18:24.247882+0000","last_undegraded":"2024-05-04T12:18:24.247882+0000","last_fullsized":"2024-05-04T12:18:24.247882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:10:47.797595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836758+0000","last_change":"2024-05-04T12:18:22.878704+0000","last_active":"2024-05-04T12:18:23.836758+0000","last_peered":"2024-05-04T12:18:23.836758+0000","last_clean":"2024-05-04T12:18:23.836758+0000","last_became_active":"2024-05-04T12:18:22.878423+0000","last_became_peered":"2024-05-04T12:18:22.878423+0000","last_unstale":"2024-05-04T12:18:23.836758+0000","last_undegraded":"2024-05-04T12:18:23.836758+0000","last_fullsized":"2024-05-04T12:18:23.836758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:28:22.639254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297059+0000","last_change":"2024-05-04T12:18:22.854847+0000","last_active":"2024-05-04T12:18:24.297059+0000","last_peered":"2024-05-04T12:18:24.297059+0000","last_clean":"2024-05-04T12:18:24.297059+0000","last_became_active":"2024-05-04T12:18:22.854509+0000","last_became_peered":"2024-05-04T12:18:22.854509+0000","last_unstale":"2024-05-04T12:18:24.297059+0000","last_undegraded":"2024-05-04T12:18:24.297059+0000","last_fullsized":"2024-05-04T12:18:24.297059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:05:46.918761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835635+0000","last_change":"2024-05-04T12:18:22.856252+0000","last_active":"2024-05-04T12:18:23.835635+0000","last_peered":"2024-05-04T12:18:23.835635+0000","last_clean":"2024-05-04T12:18:23.835635+0000","last_became_active":"2024-05-04T12:18:22.855950+0000","last_became_peered":"2024-05-04T12:18:22.855950+0000","last_unstale":"2024-05-04T12:18:23.835635+0000","last_undegraded":"2024-05-04T12:18:23.835635+0000","last_fullsized":"2024-05-04T12:18:23.835635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:10:47.797595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297649+0000","last_change":"2024-05-04T12:18:22.851916+0000","last_active":"2024-05-04T12:18:24.297649+0000","last_peered":"2024-05-04T12:18:24.297649+0000","last_clean":"2024-05-04T12:18:24.297649+0000","last_became_active":"2024-05-04T12:18:22.851291+0000","last_became_peered":"2024-05-04T12:18:22.851291+0000","last_unstale":"2024-05-04T12:18:24.297649+0000","last_undegraded":"2024-05-04T12:18:24.297649+0000","last_fullsized":"2024-05-04T12:18:24.297649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:02:37.747970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778425+0000","last_change":"2024-05-04T12:18:22.879929+0000","last_active":"2024-05-04T12:18:24.778425+0000","last_peered":"2024-05-04T12:18:24.778425+0000","last_clean":"2024-05-04T12:18:24.778425+0000","last_became_active":"2024-05-04T12:18:22.879364+0000","last_became_peered":"2024-05-04T12:18:22.879364+0000","last_unstale":"2024-05-04T12:18:24.778425+0000","last_undegraded":"2024-05-04T12:18:24.778425+0000","last_fullsized":"2024-05-04T12:18:24.778425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:51.416179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298129+0000","last_change":"2024-05-04T12:18:22.859239+0000","last_active":"2024-05-04T12:18:24.298129+0000","last_peered":"2024-05-04T12:18:24.298129+0000","last_clean":"2024-05-04T12:18:24.298129+0000","last_became_active":"2024-05-04T12:18:22.858831+0000","last_became_peered":"2024-05-04T12:18:22.858831+0000","last_unstale":"2024-05-04T12:18:24.298129+0000","last_undegraded":"2024-05-04T12:18:24.298129+0000","last_fullsized":"2024-05-04T12:18:24.298129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:09:40.457188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.859758+0000","last_change":"2024-05-04T12:18:22.853885+0000","last_active":"2024-05-04T12:18:22.859758+0000","last_peered":"2024-05-04T12:18:22.859758+0000","last_clean":"2024-05-04T12:18:22.859758+0000","last_became_active":"2024-05-04T12:18:22.853589+0000","last_became_peered":"2024-05-04T12:18:22.853589+0000","last_unstale":"2024-05-04T12:18:22.859758+0000","last_undegraded":"2024-05-04T12:18:22.859758+0000","last_fullsized":"2024-05-04T12:18:22.859758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:40.494337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.860764+0000","last_change":"2024-05-04T12:18:22.856994+0000","last_active":"2024-05-04T12:18:22.860764+0000","last_peered":"2024-05-04T12:18:22.860764+0000","last_clean":"2024-05-04T12:18:22.860764+0000","last_became_active":"2024-05-04T12:18:22.856651+0000","last_became_peered":"2024-05-04T12:18:22.856651+0000","last_unstale":"2024-05-04T12:18:22.860764+0000","last_undegraded":"2024-05-04T12:18:22.860764+0000","last_fullsized":"2024-05-04T12:18:22.860764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:09:03.032327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836324+0000","last_change":"2024-05-04T12:18:22.864231+0000","last_active":"2024-05-04T12:18:23.836324+0000","last_peered":"2024-05-04T12:18:23.836324+0000","last_clean":"2024-05-04T12:18:23.836324+0000","last_became_active":"2024-05-04T12:18:22.861558+0000","last_became_peered":"2024-05-04T12:18:22.861558+0000","last_unstale":"2024-05-04T12:18:23.836324+0000","last_undegraded":"2024-05-04T12:18:23.836324+0000","last_fullsized":"2024-05-04T12:18:23.836324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:37.729658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779374+0000","last_change":"2024-05-04T12:18:22.868189+0000","last_active":"2024-05-04T12:18:24.779374+0000","last_peered":"2024-05-04T12:18:24.779374+0000","last_clean":"2024-05-04T12:18:24.779374+0000","last_became_active":"2024-05-04T12:18:22.860456+0000","last_became_peered":"2024-05-04T12:18:22.860456+0000","last_unstale":"2024-05-04T12:18:24.779374+0000","last_undegraded":"2024-05-04T12:18:24.779374+0000","last_fullsized":"2024-05-04T12:18:24.779374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:07:22.005194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.246875+0000","last_change":"2024-05-04T12:18:22.853777+0000","last_active":"2024-05-04T12:18:24.246875+0000","last_peered":"2024-05-04T12:18:24.246875+0000","last_clean":"2024-05-04T12:18:24.246875+0000","last_became_active":"2024-05-04T12:18:22.852626+0000","last_became_peered":"2024-05-04T12:18:22.852626+0000","last_unstale":"2024-05-04T12:18:24.246875+0000","last_undegraded":"2024-05-04T12:18:24.246875+0000","last_fullsized":"2024-05-04T12:18:24.246875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.833385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831334+0000","last_change":"2024-05-04T12:18:22.844392+0000","last_active":"2024-05-04T12:18:23.831334+0000","last_peered":"2024-05-04T12:18:23.831334+0000","last_clean":"2024-05-04T12:18:23.831334+0000","last_became_active":"2024-05-04T12:18:22.843539+0000","last_became_peered":"2024-05-04T12:18:22.843539+0000","last_unstale":"2024-05-04T12:18:23.831334+0000","last_undegraded":"2024-05-04T12:18:23.831334+0000","last_fullsized":"2024-05-04T12:18:23.831334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:18:08.570317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.834671+0000","last_change":"2024-05-04T12:18:22.849846+0000","last_active":"2024-05-04T12:18:23.834671+0000","last_peered":"2024-05-04T12:18:23.834671+0000","last_clean":"2024-05-04T12:18:23.834671+0000","last_became_active":"2024-05-04T12:18:22.849552+0000","last_became_peered":"2024-05-04T12:18:22.849552+0000","last_unstale":"2024-05-04T12:18:23.834671+0000","last_undegraded":"2024-05-04T12:18:23.834671+0000","last_fullsized":"2024-05-04T12:18:23.834671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.837479+0000","last_change":"2024-05-04T12:18:22.851494+0000","last_active":"2024-05-04T12:18:23.837479+0000","last_peered":"2024-05-04T12:18:23.837479+0000","last_clean":"2024-05-04T12:18:23.837479+0000","last_became_active":"2024-05-04T12:18:22.851177+0000","last_became_peered":"2024-05-04T12:18:22.851177+0000","last_unstale":"2024-05-04T12:18:23.837479+0000","last_undegraded":"2024-05-04T12:18:23.837479+0000","last_fullsized":"2024-05-04T12:18:23.837479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:52:05.609251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.864466+0000","last_change":"2024-05-04T12:18:22.861585+0000","last_active":"2024-05-04T12:18:22.864466+0000","last_peered":"2024-05-04T12:18:22.864466+0000","last_clean":"2024-05-04T12:18:22.864466+0000","last_became_active":"2024-05-04T12:18:22.861304+0000","last_became_peered":"2024-05-04T12:18:22.861304+0000","last_unstale":"2024-05-04T12:18:22.864466+0000","last_undegraded":"2024-05-04T12:18:22.864466+0000","last_fullsized":"2024-05-04T12:18:22.864466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:59.977088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297947+0000","last_change":"2024-05-04T12:18:22.855243+0000","last_active":"2024-05-04T12:18:24.297947+0000","last_peered":"2024-05-04T12:18:24.297947+0000","last_clean":"2024-05-04T12:18:24.297947+0000","last_became_active":"2024-05-04T12:18:22.854938+0000","last_became_peered":"2024-05-04T12:18:22.854938+0000","last_unstale":"2024-05-04T12:18:24.297947+0000","last_undegraded":"2024-05-04T12:18:24.297947+0000","last_fullsized":"2024-05-04T12:18:24.297947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:52:05.609251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.874821+0000","last_change":"2024-05-04T12:18:22.870381+0000","last_active":"2024-05-04T12:18:22.874821+0000","last_peered":"2024-05-04T12:18:22.874821+0000","last_clean":"2024-05-04T12:18:22.874821+0000","last_became_active":"2024-05-04T12:18:22.870076+0000","last_became_peered":"2024-05-04T12:18:22.870076+0000","last_unstale":"2024-05-04T12:18:22.874821+0000","last_undegraded":"2024-05-04T12:18:22.874821+0000","last_fullsized":"2024-05-04T12:18:22.874821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:16:07.430915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247261+0000","last_change":"2024-05-04T12:18:22.853194+0000","last_active":"2024-05-04T12:18:24.247261+0000","last_peered":"2024-05-04T12:18:24.247261+0000","last_clean":"2024-05-04T12:18:24.247261+0000","last_became_active":"2024-05-04T12:18:22.852625+0000","last_became_peered":"2024-05-04T12:18:22.852625+0000","last_unstale":"2024-05-04T12:18:24.247261+0000","last_undegraded":"2024-05-04T12:18:24.247261+0000","last_fullsized":"2024-05-04T12:18:24.247261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:51.416179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836035+0000","last_change":"2024-05-04T12:18:22.844120+0000","last_active":"2024-05-04T12:18:23.836035+0000","last_peered":"2024-05-04T12:18:23.836035+0000","last_clean":"2024-05-04T12:18:23.836035+0000","last_became_active":"2024-05-04T12:18:22.841231+0000","last_became_peered":"2024-05-04T12:18:22.841231+0000","last_unstale":"2024-05-04T12:18:23.836035+0000","last_undegraded":"2024-05-04T12:18:23.836035+0000","last_fullsized":"2024-05-04T12:18:23.836035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:07:22.005194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778499+0000","last_change":"2024-05-04T12:18:22.877765+0000","last_active":"2024-05-04T12:18:24.778499+0000","last_peered":"2024-05-04T12:18:24.778499+0000","last_clean":"2024-05-04T12:18:24.778499+0000","last_became_active":"2024-05-04T12:18:22.877351+0000","last_became_peered":"2024-05-04T12:18:22.877351+0000","last_unstale":"2024-05-04T12:18:24.778499+0000","last_undegraded":"2024-05-04T12:18:24.778499+0000","last_fullsized":"2024-05-04T12:18:24.778499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:18:08.570317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.834935+0000","last_change":"2024-05-04T12:18:22.878164+0000","last_active":"2024-05-04T12:18:23.834935+0000","last_peered":"2024-05-04T12:18:23.834935+0000","last_clean":"2024-05-04T12:18:23.834935+0000","last_became_active":"2024-05-04T12:18:22.877881+0000","last_became_peered":"2024-05-04T12:18:22.877881+0000","last_unstale":"2024-05-04T12:18:23.834935+0000","last_undegraded":"2024-05-04T12:18:23.834935+0000","last_fullsized":"2024-05-04T12:18:23.834935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778748+0000","last_change":"2024-05-04T12:18:22.880454+0000","last_active":"2024-05-04T12:18:24.778748+0000","last_peered":"2024-05-04T12:18:24.778748+0000","last_clean":"2024-05-04T12:18:24.778748+0000","last_became_active":"2024-05-04T12:18:22.877223+0000","last_became_peered":"2024-05-04T12:18:22.877223+0000","last_unstale":"2024-05-04T12:18:24.778748+0000","last_undegraded":"2024-05-04T12:18:24.778748+0000","last_fullsized":"2024-05-04T12:18:24.778748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.636826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.868715+0000","last_change":"2024-05-04T12:18:22.862461+0000","last_active":"2024-05-04T12:18:22.868715+0000","last_peered":"2024-05-04T12:18:22.868715+0000","last_clean":"2024-05-04T12:18:22.868715+0000","last_became_active":"2024-05-04T12:18:22.862122+0000","last_became_peered":"2024-05-04T12:18:22.862122+0000","last_unstale":"2024-05-04T12:18:22.868715+0000","last_undegraded":"2024-05-04T12:18:22.868715+0000","last_fullsized":"2024-05-04T12:18:22.868715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:19:54.361385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831950+0000","last_change":"2024-05-04T12:18:22.870835+0000","last_active":"2024-05-04T12:18:23.831950+0000","last_peered":"2024-05-04T12:18:23.831950+0000","last_clean":"2024-05-04T12:18:23.831950+0000","last_became_active":"2024-05-04T12:18:22.870544+0000","last_became_peered":"2024-05-04T12:18:22.870544+0000","last_unstale":"2024-05-04T12:18:23.831950+0000","last_undegraded":"2024-05-04T12:18:23.831950+0000","last_fullsized":"2024-05-04T12:18:23.831950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779872+0000","last_change":"2024-05-04T12:18:22.869151+0000","last_active":"2024-05-04T12:18:24.779872+0000","last_peered":"2024-05-04T12:18:24.779872+0000","last_clean":"2024-05-04T12:18:24.779872+0000","last_became_active":"2024-05-04T12:18:22.868856+0000","last_became_peered":"2024-05-04T12:18:22.868856+0000","last_unstale":"2024-05-04T12:18:24.779872+0000","last_undegraded":"2024-05-04T12:18:24.779872+0000","last_fullsized":"2024-05-04T12:18:24.779872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:28:22.639254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247351+0000","last_change":"2024-05-04T12:18:22.873982+0000","last_active":"2024-05-04T12:18:24.247351+0000","last_peered":"2024-05-04T12:18:24.247351+0000","last_clean":"2024-05-04T12:18:24.247351+0000","last_became_active":"2024-05-04T12:18:22.871687+0000","last_became_peered":"2024-05-04T12:18:22.871687+0000","last_unstale":"2024-05-04T12:18:24.247351+0000","last_undegraded":"2024-05-04T12:18:24.247351+0000","last_fullsized":"2024-05-04T12:18:24.247351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:47:19.879004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836157+0000","last_change":"2024-05-04T12:18:22.878584+0000","last_active":"2024-05-04T12:18:23.836157+0000","last_peered":"2024-05-04T12:18:23.836157+0000","last_clean":"2024-05-04T12:18:23.836157+0000","last_became_active":"2024-05-04T12:18:22.878093+0000","last_became_peered":"2024-05-04T12:18:22.878093+0000","last_unstale":"2024-05-04T12:18:23.836157+0000","last_undegraded":"2024-05-04T12:18:23.836157+0000","last_fullsized":"2024-05-04T12:18:23.836157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:40:13.207623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.863872+0000","last_change":"2024-05-04T12:18:22.851028+0000","last_active":"2024-05-04T12:18:22.863872+0000","last_peered":"2024-05-04T12:18:22.863872+0000","last_clean":"2024-05-04T12:18:22.863872+0000","last_became_active":"2024-05-04T12:18:22.850280+0000","last_became_peered":"2024-05-04T12:18:22.850280+0000","last_unstale":"2024-05-04T12:18:22.863872+0000","last_undegraded":"2024-05-04T12:18:22.863872+0000","last_fullsized":"2024-05-04T12:18:22.863872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:23.487598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.874849+0000","last_change":"2024-05-04T12:18:22.870893+0000","last_active":"2024-05-04T12:18:22.874849+0000","last_peered":"2024-05-04T12:18:22.874849+0000","last_clean":"2024-05-04T12:18:22.874849+0000","last_became_active":"2024-05-04T12:18:22.870517+0000","last_became_peered":"2024-05-04T12:18:22.870517+0000","last_unstale":"2024-05-04T12:18:22.874849+0000","last_undegraded":"2024-05-04T12:18:22.874849+0000","last_fullsized":"2024-05-04T12:18:22.874849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:27:57.193126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835952+0000","last_change":"2024-05-04T12:18:22.840511+0000","last_active":"2024-05-04T12:18:23.835952+0000","last_peered":"2024-05-04T12:18:23.835952+0000","last_clean":"2024-05-04T12:18:23.835952+0000","last_became_active":"2024-05-04T12:18:22.839811+0000","last_became_peered":"2024-05-04T12:18:22.839811+0000","last_unstale":"2024-05-04T12:18:23.835952+0000","last_undegraded":"2024-05-04T12:18:23.835952+0000","last_fullsized":"2024-05-04T12:18:23.835952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:59.977088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779675+0000","last_change":"2024-05-04T12:18:21.838819+0000","last_active":"2024-05-04T12:18:24.779675+0000","last_peered":"2024-05-04T12:18:24.779675+0000","last_clean":"2024-05-04T12:18:24.779675+0000","last_became_active":"2024-05-04T12:18:21.838503+0000","last_became_peered":"2024-05-04T12:18:21.838503+0000","last_unstale":"2024-05-04T12:18:24.779675+0000","last_undegraded":"2024-05-04T12:18:24.779675+0000","last_fullsized":"2024-05-04T12:18:24.779675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247583+0000","last_change":"2024-05-04T12:18:22.854643+0000","last_active":"2024-05-04T12:18:24.247583+0000","last_peered":"2024-05-04T12:18:24.247583+0000","last_clean":"2024-05-04T12:18:24.247583+0000","last_became_active":"2024-05-04T12:18:22.854063+0000","last_became_peered":"2024-05-04T12:18:22.854063+0000","last_unstale":"2024-05-04T12:18:24.247583+0000","last_undegraded":"2024-05-04T12:18:24.247583+0000","last_fullsized":"2024-05-04T12:18:24.247583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.860055+0000","last_change":"2024-05-04T12:18:21.849103+0000","last_active":"2024-05-04T12:18:22.860055+0000","last_peered":"2024-05-04T12:18:22.860055+0000","last_clean":"2024-05-04T12:18:22.860055+0000","last_became_active":"2024-05-04T12:18:21.848433+0000","last_became_peered":"2024-05-04T12:18:21.848433+0000","last_unstale":"2024-05-04T12:18:22.860055+0000","last_undegraded":"2024-05-04T12:18:22.860055+0000","last_fullsized":"2024-05-04T12:18:22.860055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778407+0000","last_change":"2024-05-04T12:18:21.851455+0000","last_active":"2024-05-04T12:18:24.778407+0000","last_peered":"2024-05-04T12:18:24.778407+0000","last_clean":"2024-05-04T12:18:24.778407+0000","last_became_active":"2024-05-04T12:18:21.851229+0000","last_became_peered":"2024-05-04T12:18:21.851229+0000","last_unstale":"2024-05-04T12:18:24.778407+0000","last_undegraded":"2024-05-04T12:18:24.778407+0000","last_fullsized":"2024-05-04T12:18:24.778407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778619+0000","last_change":"2024-05-04T12:18:22.874715+0000","last_active":"2024-05-04T12:18:24.778619+0000","last_peered":"2024-05-04T12:18:24.778619+0000","last_clean":"2024-05-04T12:18:24.778619+0000","last_became_active":"2024-05-04T12:18:22.869353+0000","last_became_peered":"2024-05-04T12:18:22.869353+0000","last_unstale":"2024-05-04T12:18:24.778619+0000","last_undegraded":"2024-05-04T12:18:24.778619+0000","last_fullsized":"2024-05-04T12:18:24.778619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:47:19.879004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835052+0000","last_change":"2024-05-04T12:18:21.832974+0000","last_active":"2024-05-04T12:18:23.835052+0000","last_peered":"2024-05-04T12:18:23.835052+0000","last_clean":"2024-05-04T12:18:23.835052+0000","last_became_active":"2024-05-04T12:18:21.832317+0000","last_became_peered":"2024-05-04T12:18:21.832317+0000","last_unstale":"2024-05-04T12:18:23.835052+0000","last_undegraded":"2024-05-04T12:18:23.835052+0000","last_fullsized":"2024-05-04T12:18:23.835052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831070+0000","last_change":"2024-05-04T12:18:22.871132+0000","last_active":"2024-05-04T12:18:23.831070+0000","last_peered":"2024-05-04T12:18:23.831070+0000","last_clean":"2024-05-04T12:18:23.831070+0000","last_became_active":"2024-05-04T12:18:22.870622+0000","last_became_peered":"2024-05-04T12:18:22.870622+0000","last_unstale":"2024-05-04T12:18:23.831070+0000","last_undegraded":"2024-05-04T12:18:23.831070+0000","last_fullsized":"2024-05-04T12:18:23.831070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:51.416179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.848989+0000","last_change":"2024-05-04T12:18:21.846618+0000","last_active":"2024-05-04T12:18:22.848989+0000","last_peered":"2024-05-04T12:18:22.848989+0000","last_clean":"2024-05-04T12:18:22.848989+0000","last_became_active":"2024-05-04T12:18:21.845832+0000","last_became_peered":"2024-05-04T12:18:21.845832+0000","last_unstale":"2024-05-04T12:18:22.848989+0000","last_undegraded":"2024-05-04T12:18:22.848989+0000","last_fullsized":"2024-05-04T12:18:22.848989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836853+0000","last_change":"2024-05-04T12:18:22.848101+0000","last_active":"2024-05-04T12:18:23.836853+0000","last_peered":"2024-05-04T12:18:23.836853+0000","last_clean":"2024-05-04T12:18:23.836853+0000","last_became_active":"2024-05-04T12:18:22.844375+0000","last_became_peered":"2024-05-04T12:18:22.844375+0000","last_unstale":"2024-05-04T12:18:23.836853+0000","last_undegraded":"2024-05-04T12:18:23.836853+0000","last_fullsized":"2024-05-04T12:18:23.836853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:14:21.945963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836972+0000","last_change":"2024-05-04T12:18:21.847419+0000","last_active":"2024-05-04T12:18:23.836972+0000","last_peered":"2024-05-04T12:18:23.836972+0000","last_clean":"2024-05-04T12:18:23.836972+0000","last_became_active":"2024-05-04T12:18:21.846646+0000","last_became_peered":"2024-05-04T12:18:21.846646+0000","last_unstale":"2024-05-04T12:18:23.836972+0000","last_undegraded":"2024-05-04T12:18:23.836972+0000","last_fullsized":"2024-05-04T12:18:23.836972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247246+0000","last_change":"2024-05-04T12:18:22.850137+0000","last_active":"2024-05-04T12:18:24.247246+0000","last_peered":"2024-05-04T12:18:24.247246+0000","last_clean":"2024-05-04T12:18:24.247246+0000","last_became_active":"2024-05-04T12:18:22.849691+0000","last_became_peered":"2024-05-04T12:18:22.849691+0000","last_unstale":"2024-05-04T12:18:24.247246+0000","last_undegraded":"2024-05-04T12:18:24.247246+0000","last_fullsized":"2024-05-04T12:18:24.247246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:27:57.193126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.832204+0000","last_change":"2024-05-04T12:18:21.838666+0000","last_active":"2024-05-04T12:18:23.832204+0000","last_peered":"2024-05-04T12:18:23.832204+0000","last_clean":"2024-05-04T12:18:23.832204+0000","last_became_active":"2024-05-04T12:18:21.838251+0000","last_became_peered":"2024-05-04T12:18:21.838251+0000","last_unstale":"2024-05-04T12:18:23.832204+0000","last_undegraded":"2024-05-04T12:18:23.832204+0000","last_fullsized":"2024-05-04T12:18:23.832204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836726+0000","last_change":"2024-05-04T12:18:22.849782+0000","last_active":"2024-05-04T12:18:23.836726+0000","last_peered":"2024-05-04T12:18:23.836726+0000","last_clean":"2024-05-04T12:18:23.836726+0000","last_became_active":"2024-05-04T12:18:22.843535+0000","last_became_peered":"2024-05-04T12:18:22.843535+0000","last_unstale":"2024-05-04T12:18:23.836726+0000","last_undegraded":"2024-05-04T12:18:23.836726+0000","last_fullsized":"2024-05-04T12:18:23.836726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:40.494337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.885934+0000","last_change":"2024-05-04T12:18:21.844876+0000","last_active":"2024-05-04T12:18:22.885934+0000","last_peered":"2024-05-04T12:18:22.885934+0000","last_clean":"2024-05-04T12:18:22.885934+0000","last_became_active":"2024-05-04T12:18:21.844036+0000","last_became_peered":"2024-05-04T12:18:21.844036+0000","last_unstale":"2024-05-04T12:18:22.885934+0000","last_undegraded":"2024-05-04T12:18:22.885934+0000","last_fullsized":"2024-05-04T12:18:22.885934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297461+0000","last_change":"2024-05-04T12:18:22.878315+0000","last_active":"2024-05-04T12:18:24.297461+0000","last_peered":"2024-05-04T12:18:24.297461+0000","last_clean":"2024-05-04T12:18:24.297461+0000","last_became_active":"2024-05-04T12:18:22.877603+0000","last_became_peered":"2024-05-04T12:18:22.877603+0000","last_unstale":"2024-05-04T12:18:24.297461+0000","last_undegraded":"2024-05-04T12:18:24.297461+0000","last_fullsized":"2024-05-04T12:18:24.297461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:14:21.945963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247649+0000","last_change":"2024-05-04T12:18:21.829283+0000","last_active":"2024-05-04T12:18:24.247649+0000","last_peered":"2024-05-04T12:18:24.247649+0000","last_clean":"2024-05-04T12:18:24.247649+0000","last_became_active":"2024-05-04T12:18:21.828821+0000","last_became_peered":"2024-05-04T12:18:21.828821+0000","last_unstale":"2024-05-04T12:18:24.247649+0000","last_undegraded":"2024-05-04T12:18:24.247649+0000","last_fullsized":"2024-05-04T12:18:24.247649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:27:35.940816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247296+0000","last_change":"2024-05-04T12:18:22.851641+0000","last_active":"2024-05-04T12:18:24.247296+0000","last_peered":"2024-05-04T12:18:24.247296+0000","last_clean":"2024-05-04T12:18:24.247296+0000","last_became_active":"2024-05-04T12:18:22.850896+0000","last_became_peered":"2024-05-04T12:18:22.850896+0000","last_unstale":"2024-05-04T12:18:24.247296+0000","last_undegraded":"2024-05-04T12:18:24.247296+0000","last_fullsized":"2024-05-04T12:18:24.247296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:18:08.570317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836170+0000","last_change":"2024-05-04T12:18:21.837372+0000","last_active":"2024-05-04T12:18:23.836170+0000","last_peered":"2024-05-04T12:18:23.836170+0000","last_clean":"2024-05-04T12:18:23.836170+0000","last_became_active":"2024-05-04T12:18:21.836808+0000","last_became_peered":"2024-05-04T12:18:21.836808+0000","last_unstale":"2024-05-04T12:18:23.836170+0000","last_undegraded":"2024-05-04T12:18:23.836170+0000","last_fullsized":"2024-05-04T12:18:23.836170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297317+0000","last_change":"2024-05-04T12:18:22.852179+0000","last_active":"2024-05-04T12:18:24.297317+0000","last_peered":"2024-05-04T12:18:24.297317+0000","last_clean":"2024-05-04T12:18:24.297317+0000","last_became_active":"2024-05-04T12:18:22.851865+0000","last_became_peered":"2024-05-04T12:18:22.851865+0000","last_unstale":"2024-05-04T12:18:24.297317+0000","last_undegraded":"2024-05-04T12:18:24.297317+0000","last_fullsized":"2024-05-04T12:18:24.297317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:28:22.639254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.849403+0000","last_change":"2024-05-04T12:18:21.846060+0000","last_active":"2024-05-04T12:18:22.849403+0000","last_peered":"2024-05-04T12:18:22.849403+0000","last_clean":"2024-05-04T12:18:22.849403+0000","last_became_active":"2024-05-04T12:18:21.844968+0000","last_became_peered":"2024-05-04T12:18:21.844968+0000","last_unstale":"2024-05-04T12:18:22.849403+0000","last_undegraded":"2024-05-04T12:18:22.849403+0000","last_fullsized":"2024-05-04T12:18:22.849403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.874823+0000","last_change":"2024-05-04T12:18:22.870884+0000","last_active":"2024-05-04T12:18:22.874823+0000","last_peered":"2024-05-04T12:18:22.874823+0000","last_clean":"2024-05-04T12:18:22.874823+0000","last_became_active":"2024-05-04T12:18:22.870454+0000","last_became_peered":"2024-05-04T12:18:22.870454+0000","last_unstale":"2024-05-04T12:18:22.874823+0000","last_undegraded":"2024-05-04T12:18:22.874823+0000","last_fullsized":"2024-05-04T12:18:22.874823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.833385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.870596+0000","last_change":"2024-05-04T12:18:21.831214+0000","last_active":"2024-05-04T12:18:22.870596+0000","last_peered":"2024-05-04T12:18:22.870596+0000","last_clean":"2024-05-04T12:18:22.870596+0000","last_became_active":"2024-05-04T12:18:21.830771+0000","last_became_peered":"2024-05-04T12:18:21.830771+0000","last_unstale":"2024-05-04T12:18:22.870596+0000","last_undegraded":"2024-05-04T12:18:22.870596+0000","last_fullsized":"2024-05-04T12:18:22.870596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:27:35.940816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296985+0000","last_change":"2024-05-04T12:18:22.852135+0000","last_active":"2024-05-04T12:18:24.296985+0000","last_peered":"2024-05-04T12:18:24.296985+0000","last_clean":"2024-05-04T12:18:24.296985+0000","last_became_active":"2024-05-04T12:18:22.851686+0000","last_became_peered":"2024-05-04T12:18:22.851686+0000","last_unstale":"2024-05-04T12:18:24.296985+0000","last_undegraded":"2024-05-04T12:18:24.296985+0000","last_fullsized":"2024-05-04T12:18:24.296985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:40:13.207623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836318+0000","last_change":"2024-05-04T12:18:21.830621+0000","last_active":"2024-05-04T12:18:23.836318+0000","last_peered":"2024-05-04T12:18:23.836318+0000","last_clean":"2024-05-04T12:18:23.836318+0000","last_became_active":"2024-05-04T12:18:21.829681+0000","last_became_peered":"2024-05-04T12:18:21.829681+0000","last_unstale":"2024-05-04T12:18:23.836318+0000","last_undegraded":"2024-05-04T12:18:23.836318+0000","last_fullsized":"2024-05-04T12:18:23.836318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:04:24.159320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297861+0000","last_change":"2024-05-04T12:18:22.855366+0000","last_active":"2024-05-04T12:18:24.297861+0000","last_peered":"2024-05-04T12:18:24.297861+0000","last_clean":"2024-05-04T12:18:24.297861+0000","last_became_active":"2024-05-04T12:18:22.855103+0000","last_became_peered":"2024-05-04T12:18:22.855103+0000","last_unstale":"2024-05-04T12:18:24.297861+0000","last_undegraded":"2024-05-04T12:18:24.297861+0000","last_fullsized":"2024-05-04T12:18:24.297861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:00:25.310837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.878682+0000","last_change":"2024-05-04T12:18:21.832957+0000","last_active":"2024-05-04T12:18:22.878682+0000","last_peered":"2024-05-04T12:18:22.878682+0000","last_clean":"2024-05-04T12:18:22.878682+0000","last_became_active":"2024-05-04T12:18:21.832274+0000","last_became_peered":"2024-05-04T12:18:21.832274+0000","last_unstale":"2024-05-04T12:18:22.878682+0000","last_undegraded":"2024-05-04T12:18:22.878682+0000","last_fullsized":"2024-05-04T12:18:22.878682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298277+0000","last_change":"2024-05-04T12:18:22.861628+0000","last_active":"2024-05-04T12:18:24.298277+0000","last_peered":"2024-05-04T12:18:24.298277+0000","last_clean":"2024-05-04T12:18:24.298277+0000","last_became_active":"2024-05-04T12:18:22.861348+0000","last_became_peered":"2024-05-04T12:18:22.861348+0000","last_unstale":"2024-05-04T12:18:24.298277+0000","last_undegraded":"2024-05-04T12:18:24.298277+0000","last_fullsized":"2024-05-04T12:18:24.298277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:21:56.962539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.832228+0000","last_change":"2024-05-04T12:18:21.832502+0000","last_active":"2024-05-04T12:18:23.832228+0000","last_peered":"2024-05-04T12:18:23.832228+0000","last_clean":"2024-05-04T12:18:23.832228+0000","last_became_active":"2024-05-04T12:18:21.832085+0000","last_became_peered":"2024-05-04T12:18:21.832085+0000","last_unstale":"2024-05-04T12:18:23.832228+0000","last_undegraded":"2024-05-04T12:18:23.832228+0000","last_fullsized":"2024-05-04T12:18:23.832228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:39.485391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836949+0000","last_change":"2024-05-04T12:18:22.870045+0000","last_active":"2024-05-04T12:18:23.836949+0000","last_peered":"2024-05-04T12:18:23.836949+0000","last_clean":"2024-05-04T12:18:23.836949+0000","last_became_active":"2024-05-04T12:18:22.869646+0000","last_became_peered":"2024-05-04T12:18:22.869646+0000","last_unstale":"2024-05-04T12:18:23.836949+0000","last_undegraded":"2024-05-04T12:18:23.836949+0000","last_fullsized":"2024-05-04T12:18:23.836949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:02:37.747970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.840032+0000","last_change":"2024-05-04T12:18:21.839604+0000","last_active":"2024-05-04T12:18:22.840032+0000","last_peered":"2024-05-04T12:18:22.840032+0000","last_clean":"2024-05-04T12:18:22.840032+0000","last_became_active":"2024-05-04T12:18:21.839304+0000","last_became_peered":"2024-05-04T12:18:21.839304+0000","last_unstale":"2024-05-04T12:18:22.840032+0000","last_undegraded":"2024-05-04T12:18:22.840032+0000","last_fullsized":"2024-05-04T12:18:22.840032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297752+0000","last_change":"2024-05-04T12:18:22.854672+0000","last_active":"2024-05-04T12:18:24.297752+0000","last_peered":"2024-05-04T12:18:24.297752+0000","last_clean":"2024-05-04T12:18:24.297752+0000","last_became_active":"2024-05-04T12:18:22.852798+0000","last_became_peered":"2024-05-04T12:18:22.852798+0000","last_unstale":"2024-05-04T12:18:24.297752+0000","last_undegraded":"2024-05-04T12:18:24.297752+0000","last_fullsized":"2024-05-04T12:18:24.297752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:16:56.544684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296796+0000","last_change":"2024-05-04T12:18:12.644577+0000","last_active":"2024-05-04T12:18:24.296796+0000","last_peered":"2024-05-04T12:18:24.296796+0000","last_clean":"2024-05-04T12:18:24.296796+0000","last_became_active":"2024-05-04T12:18:12.640041+0000","last_became_peered":"2024-05-04T12:18:12.640041+0000","last_unstale":"2024-05-04T12:18:24.296796+0000","last_undegraded":"2024-05-04T12:18:24.296796+0000","last_fullsized":"2024-05-04T12:18:24.296796+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_clean_scrub_stamp":"2024-05-04T12:18:11.260157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:15:02.089387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836180+0000","last_change":"2024-05-04T12:18:21.837361+0000","last_active":"2024-05-04T12:18:23.836180+0000","last_peered":"2024-05-04T12:18:23.836180+0000","last_clean":"2024-05-04T12:18:23.836180+0000","last_became_active":"2024-05-04T12:18:21.836437+0000","last_became_peered":"2024-05-04T12:18:21.836437+0000","last_unstale":"2024-05-04T12:18:23.836180+0000","last_undegraded":"2024-05-04T12:18:23.836180+0000","last_fullsized":"2024-05-04T12:18:23.836180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:48:12.147734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779602+0000","last_change":"2024-05-04T12:18:22.868591+0000","last_active":"2024-05-04T12:18:24.779602+0000","last_peered":"2024-05-04T12:18:24.779602+0000","last_clean":"2024-05-04T12:18:24.779602+0000","last_became_active":"2024-05-04T12:18:22.868158+0000","last_became_peered":"2024-05-04T12:18:22.868158+0000","last_unstale":"2024-05-04T12:18:24.779602+0000","last_undegraded":"2024-05-04T12:18:24.779602+0000","last_fullsized":"2024-05-04T12:18:24.779602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:05:46.918761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296711+0000","last_change":"2024-05-04T12:18:21.857252+0000","last_active":"2024-05-04T12:18:24.296711+0000","last_peered":"2024-05-04T12:18:24.296711+0000","last_clean":"2024-05-04T12:18:24.296711+0000","last_became_active":"2024-05-04T12:18:21.855446+0000","last_became_peered":"2024-05-04T12:18:21.855446+0000","last_unstale":"2024-05-04T12:18:24.296711+0000","last_undegraded":"2024-05-04T12:18:24.296711+0000","last_fullsized":"2024-05-04T12:18:24.296711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.246868+0000","last_change":"2024-05-04T12:18:22.862302+0000","last_active":"2024-05-04T12:18:24.246868+0000","last_peered":"2024-05-04T12:18:24.246868+0000","last_clean":"2024-05-04T12:18:24.246868+0000","last_became_active":"2024-05-04T12:18:22.859627+0000","last_became_peered":"2024-05-04T12:18:22.859627+0000","last_unstale":"2024-05-04T12:18:24.246868+0000","last_undegraded":"2024-05-04T12:18:24.246868+0000","last_fullsized":"2024-05-04T12:18:24.246868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:16:07.430915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.839684+0000","last_change":"2024-05-04T12:18:21.832787+0000","last_active":"2024-05-04T12:18:22.839684+0000","last_peered":"2024-05-04T12:18:22.839684+0000","last_clean":"2024-05-04T12:18:22.839684+0000","last_became_active":"2024-05-04T12:18:21.832247+0000","last_became_peered":"2024-05-04T12:18:21.832247+0000","last_unstale":"2024-05-04T12:18:22.839684+0000","last_undegraded":"2024-05-04T12:18:22.839684+0000","last_fullsized":"2024-05-04T12:18:22.839684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:48:12.147734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831403+0000","last_change":"2024-05-04T12:18:22.863972+0000","last_active":"2024-05-04T12:18:23.831403+0000","last_peered":"2024-05-04T12:18:23.831403+0000","last_clean":"2024-05-04T12:18:23.831403+0000","last_became_active":"2024-05-04T12:18:22.863122+0000","last_became_peered":"2024-05-04T12:18:22.863122+0000","last_unstale":"2024-05-04T12:18:23.831403+0000","last_undegraded":"2024-05-04T12:18:23.831403+0000","last_fullsized":"2024-05-04T12:18:23.831403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:47:19.879004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836266+0000","last_change":"2024-05-04T12:18:21.835740+0000","last_active":"2024-05-04T12:18:23.836266+0000","last_peered":"2024-05-04T12:18:23.836266+0000","last_clean":"2024-05-04T12:18:23.836266+0000","last_became_active":"2024-05-04T12:18:21.834835+0000","last_became_peered":"2024-05-04T12:18:21.834835+0000","last_unstale":"2024-05-04T12:18:23.836266+0000","last_undegraded":"2024-05-04T12:18:23.836266+0000","last_fullsized":"2024-05-04T12:18:23.836266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835634+0000","last_change":"2024-05-04T12:18:22.856686+0000","last_active":"2024-05-04T12:18:23.835634+0000","last_peered":"2024-05-04T12:18:23.835634+0000","last_clean":"2024-05-04T12:18:23.835634+0000","last_became_active":"2024-05-04T12:18:22.855883+0000","last_became_peered":"2024-05-04T12:18:22.855883+0000","last_unstale":"2024-05-04T12:18:23.835634+0000","last_undegraded":"2024-05-04T12:18:23.835634+0000","last_fullsized":"2024-05-04T12:18:23.835634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:19.851400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831728+0000","last_change":"2024-05-04T12:18:21.831578+0000","last_active":"2024-05-04T12:18:23.831728+0000","last_peered":"2024-05-04T12:18:23.831728+0000","last_clean":"2024-05-04T12:18:23.831728+0000","last_became_active":"2024-05-04T12:18:21.830473+0000","last_became_peered":"2024-05-04T12:18:21.830473+0000","last_unstale":"2024-05-04T12:18:23.831728+0000","last_undegraded":"2024-05-04T12:18:23.831728+0000","last_fullsized":"2024-05-04T12:18:23.831728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.837084+0000","last_change":"2024-05-04T12:18:22.878341+0000","last_active":"2024-05-04T12:18:23.837084+0000","last_peered":"2024-05-04T12:18:23.837084+0000","last_clean":"2024-05-04T12:18:23.837084+0000","last_became_active":"2024-05-04T12:18:22.878015+0000","last_became_peered":"2024-05-04T12:18:22.878015+0000","last_unstale":"2024-05-04T12:18:23.837084+0000","last_undegraded":"2024-05-04T12:18:23.837084+0000","last_fullsized":"2024-05-04T12:18:23.837084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:00:25.310837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298025+0000","last_change":"2024-05-04T12:18:21.857646+0000","last_active":"2024-05-04T12:18:24.298025+0000","last_peered":"2024-05-04T12:18:24.298025+0000","last_clean":"2024-05-04T12:18:24.298025+0000","last_became_active":"2024-05-04T12:18:21.856638+0000","last_became_peered":"2024-05-04T12:18:21.856638+0000","last_unstale":"2024-05-04T12:18:24.298025+0000","last_undegraded":"2024-05-04T12:18:24.298025+0000","last_fullsized":"2024-05-04T12:18:24.298025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:09:02.023381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831578+0000","last_change":"2024-05-04T12:18:22.862113+0000","last_active":"2024-05-04T12:18:23.831578+0000","last_peered":"2024-05-04T12:18:23.831578+0000","last_clean":"2024-05-04T12:18:23.831578+0000","last_became_active":"2024-05-04T12:18:22.861682+0000","last_became_peered":"2024-05-04T12:18:22.861682+0000","last_unstale":"2024-05-04T12:18:23.831578+0000","last_undegraded":"2024-05-04T12:18:23.831578+0000","last_fullsized":"2024-05-04T12:18:23.831578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.636826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.832118+0000","last_change":"2024-05-04T12:18:21.846869+0000","last_active":"2024-05-04T12:18:23.832118+0000","last_peered":"2024-05-04T12:18:23.832118+0000","last_clean":"2024-05-04T12:18:23.832118+0000","last_became_active":"2024-05-04T12:18:21.846514+0000","last_became_peered":"2024-05-04T12:18:21.846514+0000","last_unstale":"2024-05-04T12:18:23.832118+0000","last_undegraded":"2024-05-04T12:18:23.832118+0000","last_fullsized":"2024-05-04T12:18:23.832118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779211+0000","last_change":"2024-05-04T12:18:22.879784+0000","last_active":"2024-05-04T12:18:24.779211+0000","last_peered":"2024-05-04T12:18:24.779211+0000","last_clean":"2024-05-04T12:18:24.779211+0000","last_became_active":"2024-05-04T12:18:22.877733+0000","last_became_peered":"2024-05-04T12:18:22.877733+0000","last_unstale":"2024-05-04T12:18:24.779211+0000","last_undegraded":"2024-05-04T12:18:24.779211+0000","last_fullsized":"2024-05-04T12:18:24.779211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:10:47.797595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.863213+0000","last_change":"2024-05-04T12:18:21.854368+0000","last_active":"2024-05-04T12:18:22.863213+0000","last_peered":"2024-05-04T12:18:22.863213+0000","last_clean":"2024-05-04T12:18:22.863213+0000","last_became_active":"2024-05-04T12:18:21.852721+0000","last_became_peered":"2024-05-04T12:18:21.852721+0000","last_unstale":"2024-05-04T12:18:22.863213+0000","last_undegraded":"2024-05-04T12:18:22.863213+0000","last_fullsized":"2024-05-04T12:18:22.863213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247454+0000","last_change":"2024-05-04T12:18:22.853292+0000","last_active":"2024-05-04T12:18:24.247454+0000","last_peered":"2024-05-04T12:18:24.247454+0000","last_clean":"2024-05-04T12:18:24.247454+0000","last_became_active":"2024-05-04T12:18:22.852401+0000","last_became_peered":"2024-05-04T12:18:22.852401+0000","last_unstale":"2024-05-04T12:18:24.247454+0000","last_undegraded":"2024-05-04T12:18:24.247454+0000","last_fullsized":"2024-05-04T12:18:24.247454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296652+0000","last_change":"2024-05-04T12:18:21.858135+0000","last_active":"2024-05-04T12:18:24.296652+0000","last_peered":"2024-05-04T12:18:24.296652+0000","last_clean":"2024-05-04T12:18:24.296652+0000","last_became_active":"2024-05-04T12:18:21.856430+0000","last_became_peered":"2024-05-04T12:18:21.856430+0000","last_unstale":"2024-05-04T12:18:24.296652+0000","last_undegraded":"2024-05-04T12:18:24.296652+0000","last_fullsized":"2024-05-04T12:18:24.296652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835773+0000","last_change":"2024-05-04T12:18:22.852562+0000","last_active":"2024-05-04T12:18:23.835773+0000","last_peered":"2024-05-04T12:18:23.835773+0000","last_clean":"2024-05-04T12:18:23.835773+0000","last_became_active":"2024-05-04T12:18:22.851980+0000","last_became_peered":"2024-05-04T12:18:22.851980+0000","last_unstale":"2024-05-04T12:18:23.835773+0000","last_undegraded":"2024-05-04T12:18:23.835773+0000","last_fullsized":"2024-05-04T12:18:23.835773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:23.487598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831190+0000","last_change":"2024-05-04T12:18:21.837963+0000","last_active":"2024-05-04T12:18:23.831190+0000","last_peered":"2024-05-04T12:18:23.831190+0000","last_clean":"2024-05-04T12:18:23.831190+0000","last_became_active":"2024-05-04T12:18:21.837563+0000","last_became_peered":"2024-05-04T12:18:21.837563+0000","last_unstale":"2024-05-04T12:18:23.831190+0000","last_undegraded":"2024-05-04T12:18:23.831190+0000","last_fullsized":"2024-05-04T12:18:23.831190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778827+0000","last_change":"2024-05-04T12:18:22.874277+0000","last_active":"2024-05-04T12:18:24.778827+0000","last_peered":"2024-05-04T12:18:24.778827+0000","last_clean":"2024-05-04T12:18:24.778827+0000","last_became_active":"2024-05-04T12:18:22.869298+0000","last_became_peered":"2024-05-04T12:18:22.869298+0000","last_unstale":"2024-05-04T12:18:24.778827+0000","last_undegraded":"2024-05-04T12:18:24.778827+0000","last_fullsized":"2024-05-04T12:18:24.778827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831728+0000","last_change":"2024-05-04T12:18:21.831177+0000","last_active":"2024-05-04T12:18:23.831728+0000","last_peered":"2024-05-04T12:18:23.831728+0000","last_clean":"2024-05-04T12:18:23.831728+0000","last_became_active":"2024-05-04T12:18:21.830570+0000","last_became_peered":"2024-05-04T12:18:21.830570+0000","last_unstale":"2024-05-04T12:18:23.831728+0000","last_undegraded":"2024-05-04T12:18:23.831728+0000","last_fullsized":"2024-05-04T12:18:23.831728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779529+0000","last_change":"2024-05-04T12:18:22.880190+0000","last_active":"2024-05-04T12:18:24.779529+0000","last_peered":"2024-05-04T12:18:24.779529+0000","last_clean":"2024-05-04T12:18:24.779529+0000","last_became_active":"2024-05-04T12:18:22.879801+0000","last_became_peered":"2024-05-04T12:18:22.879801+0000","last_unstale":"2024-05-04T12:18:24.779529+0000","last_undegraded":"2024-05-04T12:18:24.779529+0000","last_fullsized":"2024-05-04T12:18:24.779529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:40:13.207623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297558+0000","last_change":"2024-05-04T12:18:21.860830+0000","last_active":"2024-05-04T12:18:24.297558+0000","last_peered":"2024-05-04T12:18:24.297558+0000","last_clean":"2024-05-04T12:18:24.297558+0000","last_became_active":"2024-05-04T12:18:21.860156+0000","last_became_peered":"2024-05-04T12:18:21.860156+0000","last_unstale":"2024-05-04T12:18:24.297558+0000","last_undegraded":"2024-05-04T12:18:24.297558+0000","last_fullsized":"2024-05-04T12:18:24.297558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:39.485391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.864508+0000","last_change":"2024-05-04T12:18:22.860205+0000","last_active":"2024-05-04T12:18:22.864508+0000","last_peered":"2024-05-04T12:18:22.864508+0000","last_clean":"2024-05-04T12:18:22.864508+0000","last_became_active":"2024-05-04T12:18:22.859825+0000","last_became_peered":"2024-05-04T12:18:22.859825+0000","last_unstale":"2024-05-04T12:18:22.864508+0000","last_undegraded":"2024-05-04T12:18:22.864508+0000","last_fullsized":"2024-05-04T12:18:22.864508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:03:36.140108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.837078+0000","last_change":"2024-05-04T12:18:22.844339+0000","last_active":"2024-05-04T12:18:23.837078+0000","last_peered":"2024-05-04T12:18:23.837078+0000","last_clean":"2024-05-04T12:18:23.837078+0000","last_became_active":"2024-05-04T12:18:22.843807+0000","last_became_peered":"2024-05-04T12:18:22.843807+0000","last_unstale":"2024-05-04T12:18:23.837078+0000","last_undegraded":"2024-05-04T12:18:23.837078+0000","last_fullsized":"2024-05-04T12:18:23.837078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:16:56.544684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297584+0000","last_change":"2024-05-04T12:18:21.860670+0000","last_active":"2024-05-04T12:18:24.297584+0000","last_peered":"2024-05-04T12:18:24.297584+0000","last_clean":"2024-05-04T12:18:24.297584+0000","last_became_active":"2024-05-04T12:18:21.859826+0000","last_became_peered":"2024-05-04T12:18:21.859826+0000","last_unstale":"2024-05-04T12:18:24.297584+0000","last_undegraded":"2024-05-04T12:18:24.297584+0000","last_fullsized":"2024-05-04T12:18:24.297584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836737+0000","last_change":"2024-05-04T12:18:22.846218+0000","last_active":"2024-05-04T12:18:23.836737+0000","last_peered":"2024-05-04T12:18:23.836737+0000","last_clean":"2024-05-04T12:18:23.836737+0000","last_became_active":"2024-05-04T12:18:22.845967+0000","last_became_peered":"2024-05-04T12:18:22.845967+0000","last_unstale":"2024-05-04T12:18:23.836737+0000","last_undegraded":"2024-05-04T12:18:23.836737+0000","last_fullsized":"2024-05-04T12:18:23.836737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:05:46.918761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297709+0000","last_change":"2024-05-04T12:18:21.860069+0000","last_active":"2024-05-04T12:18:24.297709+0000","last_peered":"2024-05-04T12:18:24.297709+0000","last_clean":"2024-05-04T12:18:24.297709+0000","last_became_active":"2024-05-04T12:18:21.859583+0000","last_became_peered":"2024-05-04T12:18:21.859583+0000","last_unstale":"2024-05-04T12:18:24.297709+0000","last_undegraded":"2024-05-04T12:18:24.297709+0000","last_fullsized":"2024-05-04T12:18:24.297709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:03:35.131163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247407+0000","last_change":"2024-05-04T12:18:22.856277+0000","last_active":"2024-05-04T12:18:24.247407+0000","last_peered":"2024-05-04T12:18:24.247407+0000","last_clean":"2024-05-04T12:18:24.247407+0000","last_became_active":"2024-05-04T12:18:22.855958+0000","last_became_peered":"2024-05-04T12:18:22.855958+0000","last_unstale":"2024-05-04T12:18:24.247407+0000","last_undegraded":"2024-05-04T12:18:24.247407+0000","last_fullsized":"2024-05-04T12:18:24.247407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.636826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779181+0000","last_change":"2024-05-04T12:18:21.835661+0000","last_active":"2024-05-04T12:18:24.779181+0000","last_peered":"2024-05-04T12:18:24.779181+0000","last_clean":"2024-05-04T12:18:24.779181+0000","last_became_active":"2024-05-04T12:18:21.835299+0000","last_became_peered":"2024-05-04T12:18:21.835299+0000","last_unstale":"2024-05-04T12:18:24.779181+0000","last_undegraded":"2024-05-04T12:18:24.779181+0000","last_fullsized":"2024-05-04T12:18:24.779181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:09:02.023381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779181+0000","last_change":"2024-05-04T12:18:22.878092+0000","last_active":"2024-05-04T12:18:24.779181+0000","last_peered":"2024-05-04T12:18:24.779181+0000","last_clean":"2024-05-04T12:18:24.779181+0000","last_became_active":"2024-05-04T12:18:22.875650+0000","last_became_peered":"2024-05-04T12:18:22.875650+0000","last_unstale":"2024-05-04T12:18:24.779181+0000","last_undegraded":"2024-05-04T12:18:24.779181+0000","last_fullsized":"2024-05-04T12:18:24.779181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247663+0000","last_change":"2024-05-04T12:18:21.847981+0000","last_active":"2024-05-04T12:18:24.247663+0000","last_peered":"2024-05-04T12:18:24.247663+0000","last_clean":"2024-05-04T12:18:24.247663+0000","last_became_active":"2024-05-04T12:18:21.847556+0000","last_became_peered":"2024-05-04T12:18:21.847556+0000","last_unstale":"2024-05-04T12:18:24.247663+0000","last_undegraded":"2024-05-04T12:18:24.247663+0000","last_fullsized":"2024-05-04T12:18:24.247663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:07:48.655726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":70917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":70917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":69305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T12:18:26.922 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-04T12:18:27.070 INFO:tasks.ceph.mds.d.smithi111.stderr:2024-05-04T12:18:27.070+0000 7fa9b10dd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-04T12:18:27.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae384b1640 1 Processor -- start 2024-05-04T12:18:27.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae384b1640 1 -- start start 2024-05-04T12:18:27.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae384b1640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae300aa0c0 0x7fae300aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae384b1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fae300ccb50 0x7fae300cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae384b1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae3006d1b0 con 0x7fae3015bf50 2024-05-04T12:18:27.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae384b1640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fae3015ca40 con 0x7fae300ccb50 2024-05-04T12:18:27.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae384b1640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fae3006d490 con 0x7fae300aa0c0 2024-05-04T12:18:27.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3624d640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fae3015bf50 0x7fae3015c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51482/0 (socket says 172.21.15.52:51482) 2024-05-04T12:18:27.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3624d640 1 -- 172.21.15.52:0/3300880761 learned_addr learned my addr 172.21.15.52:0/3300880761 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:27.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae36a4e640 1 --2- 172.21.15.52:0/3300880761 >> v2:172.21.15.111:3300/0 conn(0x7fae300ccb50 0x7fae300cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae35a4c640 1 --2- 172.21.15.52:0/3300880761 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae300aa0c0 0x7fae300aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 -- 172.21.15.52:0/3300880761 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 23114109 0 0) 0x7fae3006d1b0 con 0x7fae3015bf50 2024-05-04T12:18:27.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 -- 172.21.15.52:0/3300880761 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae1c003530 con 0x7fae3015bf50 2024-05-04T12:18:27.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 -- 172.21.15.52:0/3300880761 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3794527116 0 0) 0x7fae1c003530 con 0x7fae3015bf50 2024-05-04T12:18:27.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 -- 172.21.15.52:0/3300880761 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae300aa0c0 msgr2=0x7fae300aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:27.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 --2- 172.21.15.52:0/3300880761 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae300aa0c0 0x7fae300aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 -- 172.21.15.52:0/3300880761 >> v2:172.21.15.111:3300/0 conn(0x7fae300ccb50 msgr2=0x7fae300cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:27.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 --2- 172.21.15.52:0/3300880761 >> v2:172.21.15.111:3300/0 conn(0x7fae300ccb50 0x7fae300cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 -- 172.21.15.52:0/3300880761 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3006d770 con 0x7fae3015bf50 2024-05-04T12:18:27.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 -- 172.21.15.52:0/3300880761 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7fae2c001680 con 0x7fae3015bf50 2024-05-04T12:18:27.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 -- 172.21.15.52:0/3300880761 >> v1:172.21.15.52:6789/0 conn(0x7fae3015bf50 legacy=0x7fae3015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:27.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 --2- 172.21.15.52:0/3300880761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c0039c0 0x7fae1c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 --2- 172.21.15.52:0/3300880761 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae300aa0c0 0x7fae1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 --2- 172.21.15.52:0/3300880761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae1c004d50 0x7fae1c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 -- 172.21.15.52:0/3300880761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae3015ca40 con 0x7fae1c004d50 2024-05-04T12:18:27.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 -- 172.21.15.52:0/3300880761 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fae3006d490 con 0x7fae1c0039c0 2024-05-04T12:18:27.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3524b640 1 -- 172.21.15.52:0/3300880761 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fae1c00b9d0 con 0x7fae300aa0c0 2024-05-04T12:18:27.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae36a4e640 1 --2- 172.21.15.52:0/3300880761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae1c004d50 0x7fae1c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3624d640 1 --2- 172.21.15.52:0/3300880761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c0039c0 0x7fae1c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae35a4c640 1 --2- 172.21.15.52:0/3300880761 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae300aa0c0 0x7fae1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae36a4e640 1 -- 172.21.15.52:0/3300880761 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae300aa0c0 msgr2=0x7fae1c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:27.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae36a4e640 1 --2- 172.21.15.52:0/3300880761 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae300aa0c0 0x7fae1c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae36a4e640 1 -- 172.21.15.52:0/3300880761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c0039c0 msgr2=0x7fae1c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:27.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae36a4e640 1 --2- 172.21.15.52:0/3300880761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c0039c0 0x7fae1c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae36a4e640 1 -- 172.21.15.52:0/3300880761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c003530 con 0x7fae1c004d50 2024-05-04T12:18:27.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.086+0000 7fae3624d640 1 --2- 172.21.15.52:0/3300880761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c0039c0 0x7fae1c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:27.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae36a4e640 1 --2- 172.21.15.52:0/3300880761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae1c004d50 0x7fae1c0051d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae28009fc0 tx=0x7fae2801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999c432add8cd48a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:27.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae3524b640 1 -- 172.21.15.52:0/3300880761 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae280b2070 con 0x7fae1c004d50 2024-05-04T12:18:27.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae3524b640 1 -- 172.21.15.52:0/3300880761 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae280a5cd0 con 0x7fae1c004d50 2024-05-04T12:18:27.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae3524b640 1 -- 172.21.15.52:0/3300880761 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28009c10 con 0x7fae1c004d50 2024-05-04T12:18:27.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae384b1640 1 -- 172.21.15.52:0/3300880761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae1c004d50 msgr2=0x7fae1c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:27.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae384b1640 1 --2- 172.21.15.52:0/3300880761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae1c004d50 0x7fae1c0051d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae28009fc0 tx=0x7fae2801ccc0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae36a4e640 1 -- 172.21.15.52:0/3300880761 reap_dead start 2024-05-04T12:18:27.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae384b1640 1 -- 172.21.15.52:0/3300880761 shutdown_connections 2024-05-04T12:18:27.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae384b1640 1 -- 172.21.15.52:0/3300880761 >> 172.21.15.52:0/3300880761 conn(0x7fae300b14a0 msgr2=0x7fae300b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:27.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae384b1640 1 -- 172.21.15.52:0/3300880761 shutdown_connections 2024-05-04T12:18:27.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae384b1640 1 -- 172.21.15.52:0/3300880761 wait complete. 2024-05-04T12:18:27.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae384b1640 1 Processor -- start 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae384b1640 1 -- start start 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae384b1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c004d50 0x7fae301e3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae384b1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae30072140 0x7fae301570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae384b1640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae30143560 0x7fae301439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae384b1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae3006cd60 con 0x7fae30072140 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae384b1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fae3006dd30 con 0x7fae1c004d50 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae384b1640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fae3006cee0 con 0x7fae30143560 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae35a4c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae30072140 0x7fae301570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae35a4c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae30072140 0x7fae301570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53310/0 (socket says 172.21.15.52:53310) 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.090+0000 7fae35a4c640 1 -- 172.21.15.52:0/1451646344 learned_addr learned my addr 172.21.15.52:0/1451646344 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae3624d640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c004d50 0x7fae301e3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae36a4e640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae30143560 0x7fae301439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae35a4c640 1 -- 172.21.15.52:0/1451646344 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae30143560 msgr2=0x7fae301439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae35a4c640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae30143560 0x7fae301439e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae35a4c640 1 -- 172.21.15.52:0/1451646344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c004d50 msgr2=0x7fae301e3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:27.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae35a4c640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c004d50 0x7fae301e3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae35a4c640 1 -- 172.21.15.52:0/1451646344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae301e4300 con 0x7fae30072140 2024-05-04T12:18:27.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae3624d640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c004d50 0x7fae301e3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:27.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae35a4c640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae30072140 0x7fae301570c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fae200115e0 tx=0x7fae20011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:27.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae26ffd640 1 -- 172.21.15.52:0/1451646344 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2000f8f0 con 0x7fae30072140 2024-05-04T12:18:27.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae26ffd640 1 -- 172.21.15.52:0/1451646344 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2000fa90 con 0x7fae30072140 2024-05-04T12:18:27.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae26ffd640 1 -- 172.21.15.52:0/1451646344 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2000fe40 con 0x7fae30072140 2024-05-04T12:18:27.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae384b1640 1 -- 172.21.15.52:0/1451646344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae301e45e0 con 0x7fae30072140 2024-05-04T12:18:27.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae384b1640 1 -- 172.21.15.52:0/1451646344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae301e4ba0 con 0x7fae30072140 2024-05-04T12:18:27.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae384b1640 1 -- 172.21.15.52:0/1451646344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae30062e50 con 0x7fae30072140 2024-05-04T12:18:27.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae26ffd640 1 -- 172.21.15.52:0/1451646344 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fae20015460 con 0x7fae30072140 2024-05-04T12:18:27.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae26ffd640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fae140678f0 0x7fae14069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae26ffd640 1 -- 172.21.15.52:0/1451646344 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fae200865e0 con 0x7fae30072140 2024-05-04T12:18:27.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae3624d640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fae140678f0 0x7fae14069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.094+0000 7fae3624d640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fae140678f0 0x7fae14069db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fae2c001ec0 tx=0x7fae2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:27.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.098+0000 7fae26ffd640 1 -- 172.21.15.52:0/1451646344 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fae20019030 con 0x7fae30072140 2024-05-04T12:18:27.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.262+0000 7fae384b1640 1 -- 172.21.15.52:0/1451646344 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae30075320 con 0x7fae140678f0 2024-05-04T12:18:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.278+0000 7fae26ffd640 1 -- 172.21.15.52:0/1451646344 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7fae30075320 con 0x7fae140678f0 2024-05-04T12:18:27.281 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:27.283 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-04T12:18:27.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.278+0000 7fae384b1640 1 -- 172.21.15.52:0/1451646344 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fae140678f0 msgr2=0x7fae14069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:27.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.278+0000 7fae384b1640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fae140678f0 0x7fae14069db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fae2c001ec0 tx=0x7fae2c009040 comp rx=0 tx=0).stop 2024-05-04T12:18:27.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.278+0000 7fae384b1640 1 -- 172.21.15.52:0/1451646344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae30072140 msgr2=0x7fae301570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:27.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.282+0000 7fae384b1640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae30072140 0x7fae301570c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fae200115e0 tx=0x7fae20011aa0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.282+0000 7fae384b1640 1 -- 172.21.15.52:0/1451646344 shutdown_connections 2024-05-04T12:18:27.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.282+0000 7fae384b1640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fae140678f0 0x7fae14069db0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.282+0000 7fae384b1640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae30143560 0x7fae301439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.282+0000 7fae384b1640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae30072140 0x7fae301570c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.282+0000 7fae384b1640 1 --2- 172.21.15.52:0/1451646344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c004d50 0x7fae301e3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.282+0000 7fae384b1640 1 -- 172.21.15.52:0/1451646344 >> 172.21.15.52:0/1451646344 conn(0x7fae300b14a0 msgr2=0x7fae300aa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:27.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.282+0000 7fae384b1640 1 -- 172.21.15.52:0/1451646344 shutdown_connections 2024-05-04T12:18:27.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.282+0000 7fae384b1640 1 -- 172.21.15.52:0/1451646344 wait complete. 2024-05-04T12:18:27.300 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-04T12:18:26.741229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235708,"kb_used_data":2884,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739012,"statfs":{"total":773094113280,"available":772852748288,"internally_reserved":0,"allocated":2953216,"data_stored":1771496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.936852"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298061+0000","last_change":"2024-05-04T12:18:22.859306+0000","last_active":"2024-05-04T12:18:24.298061+0000","last_peered":"2024-05-04T12:18:24.298061+0000","last_clean":"2024-05-04T12:18:24.298061+0000","last_became_active":"2024-05-04T12:18:22.859107+0000","last_became_peered":"2024-05-04T12:18:22.859107+0000","last_unstale":"2024-05-04T12:18:24.298061+0000","last_undegraded":"2024-05-04T12:18:24.298061+0000","last_fullsized":"2024-05-04T12:18:24.298061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:45:17.602528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.832155+0000","last_change":"2024-05-04T12:18:22.848885+0000","last_active":"2024-05-04T12:18:23.832155+0000","last_peered":"2024-05-04T12:18:23.832155+0000","last_clean":"2024-05-04T12:18:23.832155+0000","last_became_active":"2024-05-04T12:18:22.848320+0000","last_became_peered":"2024-05-04T12:18:22.848320+0000","last_unstale":"2024-05-04T12:18:23.832155+0000","last_undegraded":"2024-05-04T12:18:23.832155+0000","last_fullsized":"2024-05-04T12:18:23.832155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247882+0000","last_change":"2024-05-04T12:18:22.861471+0000","last_active":"2024-05-04T12:18:24.247882+0000","last_peered":"2024-05-04T12:18:24.247882+0000","last_clean":"2024-05-04T12:18:24.247882+0000","last_became_active":"2024-05-04T12:18:22.860827+0000","last_became_peered":"2024-05-04T12:18:22.860827+0000","last_unstale":"2024-05-04T12:18:24.247882+0000","last_undegraded":"2024-05-04T12:18:24.247882+0000","last_fullsized":"2024-05-04T12:18:24.247882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:10:47.797595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836758+0000","last_change":"2024-05-04T12:18:22.878704+0000","last_active":"2024-05-04T12:18:23.836758+0000","last_peered":"2024-05-04T12:18:23.836758+0000","last_clean":"2024-05-04T12:18:23.836758+0000","last_became_active":"2024-05-04T12:18:22.878423+0000","last_became_peered":"2024-05-04T12:18:22.878423+0000","last_unstale":"2024-05-04T12:18:23.836758+0000","last_undegraded":"2024-05-04T12:18:23.836758+0000","last_fullsized":"2024-05-04T12:18:23.836758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:28:22.639254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297059+0000","last_change":"2024-05-04T12:18:22.854847+0000","last_active":"2024-05-04T12:18:24.297059+0000","last_peered":"2024-05-04T12:18:24.297059+0000","last_clean":"2024-05-04T12:18:24.297059+0000","last_became_active":"2024-05-04T12:18:22.854509+0000","last_became_peered":"2024-05-04T12:18:22.854509+0000","last_unstale":"2024-05-04T12:18:24.297059+0000","last_undegraded":"2024-05-04T12:18:24.297059+0000","last_fullsized":"2024-05-04T12:18:24.297059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:05:46.918761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835635+0000","last_change":"2024-05-04T12:18:22.856252+0000","last_active":"2024-05-04T12:18:23.835635+0000","last_peered":"2024-05-04T12:18:23.835635+0000","last_clean":"2024-05-04T12:18:23.835635+0000","last_became_active":"2024-05-04T12:18:22.855950+0000","last_became_peered":"2024-05-04T12:18:22.855950+0000","last_unstale":"2024-05-04T12:18:23.835635+0000","last_undegraded":"2024-05-04T12:18:23.835635+0000","last_fullsized":"2024-05-04T12:18:23.835635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:10:47.797595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297649+0000","last_change":"2024-05-04T12:18:22.851916+0000","last_active":"2024-05-04T12:18:24.297649+0000","last_peered":"2024-05-04T12:18:24.297649+0000","last_clean":"2024-05-04T12:18:24.297649+0000","last_became_active":"2024-05-04T12:18:22.851291+0000","last_became_peered":"2024-05-04T12:18:22.851291+0000","last_unstale":"2024-05-04T12:18:24.297649+0000","last_undegraded":"2024-05-04T12:18:24.297649+0000","last_fullsized":"2024-05-04T12:18:24.297649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:02:37.747970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778425+0000","last_change":"2024-05-04T12:18:22.879929+0000","last_active":"2024-05-04T12:18:24.778425+0000","last_peered":"2024-05-04T12:18:24.778425+0000","last_clean":"2024-05-04T12:18:24.778425+0000","last_became_active":"2024-05-04T12:18:22.879364+0000","last_became_peered":"2024-05-04T12:18:22.879364+0000","last_unstale":"2024-05-04T12:18:24.778425+0000","last_undegraded":"2024-05-04T12:18:24.778425+0000","last_fullsized":"2024-05-04T12:18:24.778425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:51.416179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298129+0000","last_change":"2024-05-04T12:18:22.859239+0000","last_active":"2024-05-04T12:18:24.298129+0000","last_peered":"2024-05-04T12:18:24.298129+0000","last_clean":"2024-05-04T12:18:24.298129+0000","last_became_active":"2024-05-04T12:18:22.858831+0000","last_became_peered":"2024-05-04T12:18:22.858831+0000","last_unstale":"2024-05-04T12:18:24.298129+0000","last_undegraded":"2024-05-04T12:18:24.298129+0000","last_fullsized":"2024-05-04T12:18:24.298129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:09:40.457188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.859758+0000","last_change":"2024-05-04T12:18:22.853885+0000","last_active":"2024-05-04T12:18:22.859758+0000","last_peered":"2024-05-04T12:18:22.859758+0000","last_clean":"2024-05-04T12:18:22.859758+0000","last_became_active":"2024-05-04T12:18:22.853589+0000","last_became_peered":"2024-05-04T12:18:22.853589+0000","last_unstale":"2024-05-04T12:18:22.859758+0000","last_undegraded":"2024-05-04T12:18:22.859758+0000","last_fullsized":"2024-05-04T12:18:22.859758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:40.494337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.860764+0000","last_change":"2024-05-04T12:18:22.856994+0000","last_active":"2024-05-04T12:18:22.860764+0000","last_peered":"2024-05-04T12:18:22.860764+0000","last_clean":"2024-05-04T12:18:22.860764+0000","last_became_active":"2024-05-04T12:18:22.856651+0000","last_became_peered":"2024-05-04T12:18:22.856651+0000","last_unstale":"2024-05-04T12:18:22.860764+0000","last_undegraded":"2024-05-04T12:18:22.860764+0000","last_fullsized":"2024-05-04T12:18:22.860764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:09:03.032327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836324+0000","last_change":"2024-05-04T12:18:22.864231+0000","last_active":"2024-05-04T12:18:23.836324+0000","last_peered":"2024-05-04T12:18:23.836324+0000","last_clean":"2024-05-04T12:18:23.836324+0000","last_became_active":"2024-05-04T12:18:22.861558+0000","last_became_peered":"2024-05-04T12:18:22.861558+0000","last_unstale":"2024-05-04T12:18:23.836324+0000","last_undegraded":"2024-05-04T12:18:23.836324+0000","last_fullsized":"2024-05-04T12:18:23.836324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:37.729658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779374+0000","last_change":"2024-05-04T12:18:22.868189+0000","last_active":"2024-05-04T12:18:24.779374+0000","last_peered":"2024-05-04T12:18:24.779374+0000","last_clean":"2024-05-04T12:18:24.779374+0000","last_became_active":"2024-05-04T12:18:22.860456+0000","last_became_peered":"2024-05-04T12:18:22.860456+0000","last_unstale":"2024-05-04T12:18:24.779374+0000","last_undegraded":"2024-05-04T12:18:24.779374+0000","last_fullsized":"2024-05-04T12:18:24.779374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:07:22.005194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.246875+0000","last_change":"2024-05-04T12:18:22.853777+0000","last_active":"2024-05-04T12:18:24.246875+0000","last_peered":"2024-05-04T12:18:24.246875+0000","last_clean":"2024-05-04T12:18:24.246875+0000","last_became_active":"2024-05-04T12:18:22.852626+0000","last_became_peered":"2024-05-04T12:18:22.852626+0000","last_unstale":"2024-05-04T12:18:24.246875+0000","last_undegraded":"2024-05-04T12:18:24.246875+0000","last_fullsized":"2024-05-04T12:18:24.246875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.833385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831334+0000","last_change":"2024-05-04T12:18:22.844392+0000","last_active":"2024-05-04T12:18:23.831334+0000","last_peered":"2024-05-04T12:18:23.831334+0000","last_clean":"2024-05-04T12:18:23.831334+0000","last_became_active":"2024-05-04T12:18:22.843539+0000","last_became_peered":"2024-05-04T12:18:22.843539+0000","last_unstale":"2024-05-04T12:18:23.831334+0000","last_undegraded":"2024-05-04T12:18:23.831334+0000","last_fullsized":"2024-05-04T12:18:23.831334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:18:08.570317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.834671+0000","last_change":"2024-05-04T12:18:22.849846+0000","last_active":"2024-05-04T12:18:23.834671+0000","last_peered":"2024-05-04T12:18:23.834671+0000","last_clean":"2024-05-04T12:18:23.834671+0000","last_became_active":"2024-05-04T12:18:22.849552+0000","last_became_peered":"2024-05-04T12:18:22.849552+0000","last_unstale":"2024-05-04T12:18:23.834671+0000","last_undegraded":"2024-05-04T12:18:23.834671+0000","last_fullsized":"2024-05-04T12:18:23.834671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.837479+0000","last_change":"2024-05-04T12:18:22.851494+0000","last_active":"2024-05-04T12:18:23.837479+0000","last_peered":"2024-05-04T12:18:23.837479+0000","last_clean":"2024-05-04T12:18:23.837479+0000","last_became_active":"2024-05-04T12:18:22.851177+0000","last_became_peered":"2024-05-04T12:18:22.851177+0000","last_unstale":"2024-05-04T12:18:23.837479+0000","last_undegraded":"2024-05-04T12:18:23.837479+0000","last_fullsized":"2024-05-04T12:18:23.837479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:52:05.609251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.864466+0000","last_change":"2024-05-04T12:18:22.861585+0000","last_active":"2024-05-04T12:18:22.864466+0000","last_peered":"2024-05-04T12:18:22.864466+0000","last_clean":"2024-05-04T12:18:22.864466+0000","last_became_active":"2024-05-04T12:18:22.861304+0000","last_became_peered":"2024-05-04T12:18:22.861304+0000","last_unstale":"2024-05-04T12:18:22.864466+0000","last_undegraded":"2024-05-04T12:18:22.864466+0000","last_fullsized":"2024-05-04T12:18:22.864466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:59.977088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297947+0000","last_change":"2024-05-04T12:18:22.855243+0000","last_active":"2024-05-04T12:18:24.297947+0000","last_peered":"2024-05-04T12:18:24.297947+0000","last_clean":"2024-05-04T12:18:24.297947+0000","last_became_active":"2024-05-04T12:18:22.854938+0000","last_became_peered":"2024-05-04T12:18:22.854938+0000","last_unstale":"2024-05-04T12:18:24.297947+0000","last_undegraded":"2024-05-04T12:18:24.297947+0000","last_fullsized":"2024-05-04T12:18:24.297947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:52:05.609251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.874821+0000","last_change":"2024-05-04T12:18:22.870381+0000","last_active":"2024-05-04T12:18:22.874821+0000","last_peered":"2024-05-04T12:18:22.874821+0000","last_clean":"2024-05-04T12:18:22.874821+0000","last_became_active":"2024-05-04T12:18:22.870076+0000","last_became_peered":"2024-05-04T12:18:22.870076+0000","last_unstale":"2024-05-04T12:18:22.874821+0000","last_undegraded":"2024-05-04T12:18:22.874821+0000","last_fullsized":"2024-05-04T12:18:22.874821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:16:07.430915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247261+0000","last_change":"2024-05-04T12:18:22.853194+0000","last_active":"2024-05-04T12:18:24.247261+0000","last_peered":"2024-05-04T12:18:24.247261+0000","last_clean":"2024-05-04T12:18:24.247261+0000","last_became_active":"2024-05-04T12:18:22.852625+0000","last_became_peered":"2024-05-04T12:18:22.852625+0000","last_unstale":"2024-05-04T12:18:24.247261+0000","last_undegraded":"2024-05-04T12:18:24.247261+0000","last_fullsized":"2024-05-04T12:18:24.247261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:51.416179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836035+0000","last_change":"2024-05-04T12:18:22.844120+0000","last_active":"2024-05-04T12:18:23.836035+0000","last_peered":"2024-05-04T12:18:23.836035+0000","last_clean":"2024-05-04T12:18:23.836035+0000","last_became_active":"2024-05-04T12:18:22.841231+0000","last_became_peered":"2024-05-04T12:18:22.841231+0000","last_unstale":"2024-05-04T12:18:23.836035+0000","last_undegraded":"2024-05-04T12:18:23.836035+0000","last_fullsized":"2024-05-04T12:18:23.836035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:07:22.005194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778499+0000","last_change":"2024-05-04T12:18:22.877765+0000","last_active":"2024-05-04T12:18:24.778499+0000","last_peered":"2024-05-04T12:18:24.778499+0000","last_clean":"2024-05-04T12:18:24.778499+0000","last_became_active":"2024-05-04T12:18:22.877351+0000","last_became_peered":"2024-05-04T12:18:22.877351+0000","last_unstale":"2024-05-04T12:18:24.778499+0000","last_undegraded":"2024-05-04T12:18:24.778499+0000","last_fullsized":"2024-05-04T12:18:24.778499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:18:08.570317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.834935+0000","last_change":"2024-05-04T12:18:22.878164+0000","last_active":"2024-05-04T12:18:23.834935+0000","last_peered":"2024-05-04T12:18:23.834935+0000","last_clean":"2024-05-04T12:18:23.834935+0000","last_became_active":"2024-05-04T12:18:22.877881+0000","last_became_peered":"2024-05-04T12:18:22.877881+0000","last_unstale":"2024-05-04T12:18:23.834935+0000","last_undegraded":"2024-05-04T12:18:23.834935+0000","last_fullsized":"2024-05-04T12:18:23.834935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778748+0000","last_change":"2024-05-04T12:18:22.880454+0000","last_active":"2024-05-04T12:18:24.778748+0000","last_peered":"2024-05-04T12:18:24.778748+0000","last_clean":"2024-05-04T12:18:24.778748+0000","last_became_active":"2024-05-04T12:18:22.877223+0000","last_became_peered":"2024-05-04T12:18:22.877223+0000","last_unstale":"2024-05-04T12:18:24.778748+0000","last_undegraded":"2024-05-04T12:18:24.778748+0000","last_fullsized":"2024-05-04T12:18:24.778748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.636826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.868715+0000","last_change":"2024-05-04T12:18:22.862461+0000","last_active":"2024-05-04T12:18:22.868715+0000","last_peered":"2024-05-04T12:18:22.868715+0000","last_clean":"2024-05-04T12:18:22.868715+0000","last_became_active":"2024-05-04T12:18:22.862122+0000","last_became_peered":"2024-05-04T12:18:22.862122+0000","last_unstale":"2024-05-04T12:18:22.868715+0000","last_undegraded":"2024-05-04T12:18:22.868715+0000","last_fullsized":"2024-05-04T12:18:22.868715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:19:54.361385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831950+0000","last_change":"2024-05-04T12:18:22.870835+0000","last_active":"2024-05-04T12:18:23.831950+0000","last_peered":"2024-05-04T12:18:23.831950+0000","last_clean":"2024-05-04T12:18:23.831950+0000","last_became_active":"2024-05-04T12:18:22.870544+0000","last_became_peered":"2024-05-04T12:18:22.870544+0000","last_unstale":"2024-05-04T12:18:23.831950+0000","last_undegraded":"2024-05-04T12:18:23.831950+0000","last_fullsized":"2024-05-04T12:18:23.831950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779872+0000","last_change":"2024-05-04T12:18:22.869151+0000","last_active":"2024-05-04T12:18:24.779872+0000","last_peered":"2024-05-04T12:18:24.779872+0000","last_clean":"2024-05-04T12:18:24.779872+0000","last_became_active":"2024-05-04T12:18:22.868856+0000","last_became_peered":"2024-05-04T12:18:22.868856+0000","last_unstale":"2024-05-04T12:18:24.779872+0000","last_undegraded":"2024-05-04T12:18:24.779872+0000","last_fullsized":"2024-05-04T12:18:24.779872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:28:22.639254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247351+0000","last_change":"2024-05-04T12:18:22.873982+0000","last_active":"2024-05-04T12:18:24.247351+0000","last_peered":"2024-05-04T12:18:24.247351+0000","last_clean":"2024-05-04T12:18:24.247351+0000","last_became_active":"2024-05-04T12:18:22.871687+0000","last_became_peered":"2024-05-04T12:18:22.871687+0000","last_unstale":"2024-05-04T12:18:24.247351+0000","last_undegraded":"2024-05-04T12:18:24.247351+0000","last_fullsized":"2024-05-04T12:18:24.247351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:47:19.879004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836157+0000","last_change":"2024-05-04T12:18:22.878584+0000","last_active":"2024-05-04T12:18:23.836157+0000","last_peered":"2024-05-04T12:18:23.836157+0000","last_clean":"2024-05-04T12:18:23.836157+0000","last_became_active":"2024-05-04T12:18:22.878093+0000","last_became_peered":"2024-05-04T12:18:22.878093+0000","last_unstale":"2024-05-04T12:18:23.836157+0000","last_undegraded":"2024-05-04T12:18:23.836157+0000","last_fullsized":"2024-05-04T12:18:23.836157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:40:13.207623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.863872+0000","last_change":"2024-05-04T12:18:22.851028+0000","last_active":"2024-05-04T12:18:22.863872+0000","last_peered":"2024-05-04T12:18:22.863872+0000","last_clean":"2024-05-04T12:18:22.863872+0000","last_became_active":"2024-05-04T12:18:22.850280+0000","last_became_peered":"2024-05-04T12:18:22.850280+0000","last_unstale":"2024-05-04T12:18:22.863872+0000","last_undegraded":"2024-05-04T12:18:22.863872+0000","last_fullsized":"2024-05-04T12:18:22.863872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:23.487598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.874849+0000","last_change":"2024-05-04T12:18:22.870893+0000","last_active":"2024-05-04T12:18:22.874849+0000","last_peered":"2024-05-04T12:18:22.874849+0000","last_clean":"2024-05-04T12:18:22.874849+0000","last_became_active":"2024-05-04T12:18:22.870517+0000","last_became_peered":"2024-05-04T12:18:22.870517+0000","last_unstale":"2024-05-04T12:18:22.874849+0000","last_undegraded":"2024-05-04T12:18:22.874849+0000","last_fullsized":"2024-05-04T12:18:22.874849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:27:57.193126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835952+0000","last_change":"2024-05-04T12:18:22.840511+0000","last_active":"2024-05-04T12:18:23.835952+0000","last_peered":"2024-05-04T12:18:23.835952+0000","last_clean":"2024-05-04T12:18:23.835952+0000","last_became_active":"2024-05-04T12:18:22.839811+0000","last_became_peered":"2024-05-04T12:18:22.839811+0000","last_unstale":"2024-05-04T12:18:23.835952+0000","last_undegraded":"2024-05-04T12:18:23.835952+0000","last_fullsized":"2024-05-04T12:18:23.835952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:59.977088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779675+0000","last_change":"2024-05-04T12:18:21.838819+0000","last_active":"2024-05-04T12:18:24.779675+0000","last_peered":"2024-05-04T12:18:24.779675+0000","last_clean":"2024-05-04T12:18:24.779675+0000","last_became_active":"2024-05-04T12:18:21.838503+0000","last_became_peered":"2024-05-04T12:18:21.838503+0000","last_unstale":"2024-05-04T12:18:24.779675+0000","last_undegraded":"2024-05-04T12:18:24.779675+0000","last_fullsized":"2024-05-04T12:18:24.779675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247583+0000","last_change":"2024-05-04T12:18:22.854643+0000","last_active":"2024-05-04T12:18:24.247583+0000","last_peered":"2024-05-04T12:18:24.247583+0000","last_clean":"2024-05-04T12:18:24.247583+0000","last_became_active":"2024-05-04T12:18:22.854063+0000","last_became_peered":"2024-05-04T12:18:22.854063+0000","last_unstale":"2024-05-04T12:18:24.247583+0000","last_undegraded":"2024-05-04T12:18:24.247583+0000","last_fullsized":"2024-05-04T12:18:24.247583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.860055+0000","last_change":"2024-05-04T12:18:21.849103+0000","last_active":"2024-05-04T12:18:22.860055+0000","last_peered":"2024-05-04T12:18:22.860055+0000","last_clean":"2024-05-04T12:18:22.860055+0000","last_became_active":"2024-05-04T12:18:21.848433+0000","last_became_peered":"2024-05-04T12:18:21.848433+0000","last_unstale":"2024-05-04T12:18:22.860055+0000","last_undegraded":"2024-05-04T12:18:22.860055+0000","last_fullsized":"2024-05-04T12:18:22.860055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778407+0000","last_change":"2024-05-04T12:18:21.851455+0000","last_active":"2024-05-04T12:18:24.778407+0000","last_peered":"2024-05-04T12:18:24.778407+0000","last_clean":"2024-05-04T12:18:24.778407+0000","last_became_active":"2024-05-04T12:18:21.851229+0000","last_became_peered":"2024-05-04T12:18:21.851229+0000","last_unstale":"2024-05-04T12:18:24.778407+0000","last_undegraded":"2024-05-04T12:18:24.778407+0000","last_fullsized":"2024-05-04T12:18:24.778407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778619+0000","last_change":"2024-05-04T12:18:22.874715+0000","last_active":"2024-05-04T12:18:24.778619+0000","last_peered":"2024-05-04T12:18:24.778619+0000","last_clean":"2024-05-04T12:18:24.778619+0000","last_became_active":"2024-05-04T12:18:22.869353+0000","last_became_peered":"2024-05-04T12:18:22.869353+0000","last_unstale":"2024-05-04T12:18:24.778619+0000","last_undegraded":"2024-05-04T12:18:24.778619+0000","last_fullsized":"2024-05-04T12:18:24.778619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:47:19.879004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835052+0000","last_change":"2024-05-04T12:18:21.832974+0000","last_active":"2024-05-04T12:18:23.835052+0000","last_peered":"2024-05-04T12:18:23.835052+0000","last_clean":"2024-05-04T12:18:23.835052+0000","last_became_active":"2024-05-04T12:18:21.832317+0000","last_became_peered":"2024-05-04T12:18:21.832317+0000","last_unstale":"2024-05-04T12:18:23.835052+0000","last_undegraded":"2024-05-04T12:18:23.835052+0000","last_fullsized":"2024-05-04T12:18:23.835052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831070+0000","last_change":"2024-05-04T12:18:22.871132+0000","last_active":"2024-05-04T12:18:23.831070+0000","last_peered":"2024-05-04T12:18:23.831070+0000","last_clean":"2024-05-04T12:18:23.831070+0000","last_became_active":"2024-05-04T12:18:22.870622+0000","last_became_peered":"2024-05-04T12:18:22.870622+0000","last_unstale":"2024-05-04T12:18:23.831070+0000","last_undegraded":"2024-05-04T12:18:23.831070+0000","last_fullsized":"2024-05-04T12:18:23.831070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:51.416179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.848989+0000","last_change":"2024-05-04T12:18:21.846618+0000","last_active":"2024-05-04T12:18:22.848989+0000","last_peered":"2024-05-04T12:18:22.848989+0000","last_clean":"2024-05-04T12:18:22.848989+0000","last_became_active":"2024-05-04T12:18:21.845832+0000","last_became_peered":"2024-05-04T12:18:21.845832+0000","last_unstale":"2024-05-04T12:18:22.848989+0000","last_undegraded":"2024-05-04T12:18:22.848989+0000","last_fullsized":"2024-05-04T12:18:22.848989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836853+0000","last_change":"2024-05-04T12:18:22.848101+0000","last_active":"2024-05-04T12:18:23.836853+0000","last_peered":"2024-05-04T12:18:23.836853+0000","last_clean":"2024-05-04T12:18:23.836853+0000","last_became_active":"2024-05-04T12:18:22.844375+0000","last_became_peered":"2024-05-04T12:18:22.844375+0000","last_unstale":"2024-05-04T12:18:23.836853+0000","last_undegraded":"2024-05-04T12:18:23.836853+0000","last_fullsized":"2024-05-04T12:18:23.836853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:14:21.945963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836972+0000","last_change":"2024-05-04T12:18:21.847419+0000","last_active":"2024-05-04T12:18:23.836972+0000","last_peered":"2024-05-04T12:18:23.836972+0000","last_clean":"2024-05-04T12:18:23.836972+0000","last_became_active":"2024-05-04T12:18:21.846646+0000","last_became_peered":"2024-05-04T12:18:21.846646+0000","last_unstale":"2024-05-04T12:18:23.836972+0000","last_undegraded":"2024-05-04T12:18:23.836972+0000","last_fullsized":"2024-05-04T12:18:23.836972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247246+0000","last_change":"2024-05-04T12:18:22.850137+0000","last_active":"2024-05-04T12:18:24.247246+0000","last_peered":"2024-05-04T12:18:24.247246+0000","last_clean":"2024-05-04T12:18:24.247246+0000","last_became_active":"2024-05-04T12:18:22.849691+0000","last_became_peered":"2024-05-04T12:18:22.849691+0000","last_unstale":"2024-05-04T12:18:24.247246+0000","last_undegraded":"2024-05-04T12:18:24.247246+0000","last_fullsized":"2024-05-04T12:18:24.247246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:27:57.193126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.832204+0000","last_change":"2024-05-04T12:18:21.838666+0000","last_active":"2024-05-04T12:18:23.832204+0000","last_peered":"2024-05-04T12:18:23.832204+0000","last_clean":"2024-05-04T12:18:23.832204+0000","last_became_active":"2024-05-04T12:18:21.838251+0000","last_became_peered":"2024-05-04T12:18:21.838251+0000","last_unstale":"2024-05-04T12:18:23.832204+0000","last_undegraded":"2024-05-04T12:18:23.832204+0000","last_fullsized":"2024-05-04T12:18:23.832204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836726+0000","last_change":"2024-05-04T12:18:22.849782+0000","last_active":"2024-05-04T12:18:23.836726+0000","last_peered":"2024-05-04T12:18:23.836726+0000","last_clean":"2024-05-04T12:18:23.836726+0000","last_became_active":"2024-05-04T12:18:22.843535+0000","last_became_peered":"2024-05-04T12:18:22.843535+0000","last_unstale":"2024-05-04T12:18:23.836726+0000","last_undegraded":"2024-05-04T12:18:23.836726+0000","last_fullsized":"2024-05-04T12:18:23.836726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:40.494337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.885934+0000","last_change":"2024-05-04T12:18:21.844876+0000","last_active":"2024-05-04T12:18:22.885934+0000","last_peered":"2024-05-04T12:18:22.885934+0000","last_clean":"2024-05-04T12:18:22.885934+0000","last_became_active":"2024-05-04T12:18:21.844036+0000","last_became_peered":"2024-05-04T12:18:21.844036+0000","last_unstale":"2024-05-04T12:18:22.885934+0000","last_undegraded":"2024-05-04T12:18:22.885934+0000","last_fullsized":"2024-05-04T12:18:22.885934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297461+0000","last_change":"2024-05-04T12:18:22.878315+0000","last_active":"2024-05-04T12:18:24.297461+0000","last_peered":"2024-05-04T12:18:24.297461+0000","last_clean":"2024-05-04T12:18:24.297461+0000","last_became_active":"2024-05-04T12:18:22.877603+0000","last_became_peered":"2024-05-04T12:18:22.877603+0000","last_unstale":"2024-05-04T12:18:24.297461+0000","last_undegraded":"2024-05-04T12:18:24.297461+0000","last_fullsized":"2024-05-04T12:18:24.297461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:14:21.945963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247649+0000","last_change":"2024-05-04T12:18:21.829283+0000","last_active":"2024-05-04T12:18:24.247649+0000","last_peered":"2024-05-04T12:18:24.247649+0000","last_clean":"2024-05-04T12:18:24.247649+0000","last_became_active":"2024-05-04T12:18:21.828821+0000","last_became_peered":"2024-05-04T12:18:21.828821+0000","last_unstale":"2024-05-04T12:18:24.247649+0000","last_undegraded":"2024-05-04T12:18:24.247649+0000","last_fullsized":"2024-05-04T12:18:24.247649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:27:35.940816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247296+0000","last_change":"2024-05-04T12:18:22.851641+0000","last_active":"2024-05-04T12:18:24.247296+0000","last_peered":"2024-05-04T12:18:24.247296+0000","last_clean":"2024-05-04T12:18:24.247296+0000","last_became_active":"2024-05-04T12:18:22.850896+0000","last_became_peered":"2024-05-04T12:18:22.850896+0000","last_unstale":"2024-05-04T12:18:24.247296+0000","last_undegraded":"2024-05-04T12:18:24.247296+0000","last_fullsized":"2024-05-04T12:18:24.247296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:18:08.570317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836170+0000","last_change":"2024-05-04T12:18:21.837372+0000","last_active":"2024-05-04T12:18:23.836170+0000","last_peered":"2024-05-04T12:18:23.836170+0000","last_clean":"2024-05-04T12:18:23.836170+0000","last_became_active":"2024-05-04T12:18:21.836808+0000","last_became_peered":"2024-05-04T12:18:21.836808+0000","last_unstale":"2024-05-04T12:18:23.836170+0000","last_undegraded":"2024-05-04T12:18:23.836170+0000","last_fullsized":"2024-05-04T12:18:23.836170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297317+0000","last_change":"2024-05-04T12:18:22.852179+0000","last_active":"2024-05-04T12:18:24.297317+0000","last_peered":"2024-05-04T12:18:24.297317+0000","last_clean":"2024-05-04T12:18:24.297317+0000","last_became_active":"2024-05-04T12:18:22.851865+0000","last_became_peered":"2024-05-04T12:18:22.851865+0000","last_unstale":"2024-05-04T12:18:24.297317+0000","last_undegraded":"2024-05-04T12:18:24.297317+0000","last_fullsized":"2024-05-04T12:18:24.297317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:28:22.639254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.849403+0000","last_change":"2024-05-04T12:18:21.846060+0000","last_active":"2024-05-04T12:18:22.849403+0000","last_peered":"2024-05-04T12:18:22.849403+0000","last_clean":"2024-05-04T12:18:22.849403+0000","last_became_active":"2024-05-04T12:18:21.844968+0000","last_became_peered":"2024-05-04T12:18:21.844968+0000","last_unstale":"2024-05-04T12:18:22.849403+0000","last_undegraded":"2024-05-04T12:18:22.849403+0000","last_fullsized":"2024-05-04T12:18:22.849403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.874823+0000","last_change":"2024-05-04T12:18:22.870884+0000","last_active":"2024-05-04T12:18:22.874823+0000","last_peered":"2024-05-04T12:18:22.874823+0000","last_clean":"2024-05-04T12:18:22.874823+0000","last_became_active":"2024-05-04T12:18:22.870454+0000","last_became_peered":"2024-05-04T12:18:22.870454+0000","last_unstale":"2024-05-04T12:18:22.874823+0000","last_undegraded":"2024-05-04T12:18:22.874823+0000","last_fullsized":"2024-05-04T12:18:22.874823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.833385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.870596+0000","last_change":"2024-05-04T12:18:21.831214+0000","last_active":"2024-05-04T12:18:22.870596+0000","last_peered":"2024-05-04T12:18:22.870596+0000","last_clean":"2024-05-04T12:18:22.870596+0000","last_became_active":"2024-05-04T12:18:21.830771+0000","last_became_peered":"2024-05-04T12:18:21.830771+0000","last_unstale":"2024-05-04T12:18:22.870596+0000","last_undegraded":"2024-05-04T12:18:22.870596+0000","last_fullsized":"2024-05-04T12:18:22.870596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:27:35.940816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296985+0000","last_change":"2024-05-04T12:18:22.852135+0000","last_active":"2024-05-04T12:18:24.296985+0000","last_peered":"2024-05-04T12:18:24.296985+0000","last_clean":"2024-05-04T12:18:24.296985+0000","last_became_active":"2024-05-04T12:18:22.851686+0000","last_became_peered":"2024-05-04T12:18:22.851686+0000","last_unstale":"2024-05-04T12:18:24.296985+0000","last_undegraded":"2024-05-04T12:18:24.296985+0000","last_fullsized":"2024-05-04T12:18:24.296985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:40:13.207623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836318+0000","last_change":"2024-05-04T12:18:21.830621+0000","last_active":"2024-05-04T12:18:23.836318+0000","last_peered":"2024-05-04T12:18:23.836318+0000","last_clean":"2024-05-04T12:18:23.836318+0000","last_became_active":"2024-05-04T12:18:21.829681+0000","last_became_peered":"2024-05-04T12:18:21.829681+0000","last_unstale":"2024-05-04T12:18:23.836318+0000","last_undegraded":"2024-05-04T12:18:23.836318+0000","last_fullsized":"2024-05-04T12:18:23.836318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:04:24.159320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297861+0000","last_change":"2024-05-04T12:18:22.855366+0000","last_active":"2024-05-04T12:18:24.297861+0000","last_peered":"2024-05-04T12:18:24.297861+0000","last_clean":"2024-05-04T12:18:24.297861+0000","last_became_active":"2024-05-04T12:18:22.855103+0000","last_became_peered":"2024-05-04T12:18:22.855103+0000","last_unstale":"2024-05-04T12:18:24.297861+0000","last_undegraded":"2024-05-04T12:18:24.297861+0000","last_fullsized":"2024-05-04T12:18:24.297861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:00:25.310837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.878682+0000","last_change":"2024-05-04T12:18:21.832957+0000","last_active":"2024-05-04T12:18:22.878682+0000","last_peered":"2024-05-04T12:18:22.878682+0000","last_clean":"2024-05-04T12:18:22.878682+0000","last_became_active":"2024-05-04T12:18:21.832274+0000","last_became_peered":"2024-05-04T12:18:21.832274+0000","last_unstale":"2024-05-04T12:18:22.878682+0000","last_undegraded":"2024-05-04T12:18:22.878682+0000","last_fullsized":"2024-05-04T12:18:22.878682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298277+0000","last_change":"2024-05-04T12:18:22.861628+0000","last_active":"2024-05-04T12:18:24.298277+0000","last_peered":"2024-05-04T12:18:24.298277+0000","last_clean":"2024-05-04T12:18:24.298277+0000","last_became_active":"2024-05-04T12:18:22.861348+0000","last_became_peered":"2024-05-04T12:18:22.861348+0000","last_unstale":"2024-05-04T12:18:24.298277+0000","last_undegraded":"2024-05-04T12:18:24.298277+0000","last_fullsized":"2024-05-04T12:18:24.298277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:21:56.962539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.832228+0000","last_change":"2024-05-04T12:18:21.832502+0000","last_active":"2024-05-04T12:18:23.832228+0000","last_peered":"2024-05-04T12:18:23.832228+0000","last_clean":"2024-05-04T12:18:23.832228+0000","last_became_active":"2024-05-04T12:18:21.832085+0000","last_became_peered":"2024-05-04T12:18:21.832085+0000","last_unstale":"2024-05-04T12:18:23.832228+0000","last_undegraded":"2024-05-04T12:18:23.832228+0000","last_fullsized":"2024-05-04T12:18:23.832228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:39.485391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836949+0000","last_change":"2024-05-04T12:18:22.870045+0000","last_active":"2024-05-04T12:18:23.836949+0000","last_peered":"2024-05-04T12:18:23.836949+0000","last_clean":"2024-05-04T12:18:23.836949+0000","last_became_active":"2024-05-04T12:18:22.869646+0000","last_became_peered":"2024-05-04T12:18:22.869646+0000","last_unstale":"2024-05-04T12:18:23.836949+0000","last_undegraded":"2024-05-04T12:18:23.836949+0000","last_fullsized":"2024-05-04T12:18:23.836949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:02:37.747970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.840032+0000","last_change":"2024-05-04T12:18:21.839604+0000","last_active":"2024-05-04T12:18:22.840032+0000","last_peered":"2024-05-04T12:18:22.840032+0000","last_clean":"2024-05-04T12:18:22.840032+0000","last_became_active":"2024-05-04T12:18:21.839304+0000","last_became_peered":"2024-05-04T12:18:21.839304+0000","last_unstale":"2024-05-04T12:18:22.840032+0000","last_undegraded":"2024-05-04T12:18:22.840032+0000","last_fullsized":"2024-05-04T12:18:22.840032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297752+0000","last_change":"2024-05-04T12:18:22.854672+0000","last_active":"2024-05-04T12:18:24.297752+0000","last_peered":"2024-05-04T12:18:24.297752+0000","last_clean":"2024-05-04T12:18:24.297752+0000","last_became_active":"2024-05-04T12:18:22.852798+0000","last_became_peered":"2024-05-04T12:18:22.852798+0000","last_unstale":"2024-05-04T12:18:24.297752+0000","last_undegraded":"2024-05-04T12:18:24.297752+0000","last_fullsized":"2024-05-04T12:18:24.297752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:16:56.544684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296796+0000","last_change":"2024-05-04T12:18:12.644577+0000","last_active":"2024-05-04T12:18:24.296796+0000","last_peered":"2024-05-04T12:18:24.296796+0000","last_clean":"2024-05-04T12:18:24.296796+0000","last_became_active":"2024-05-04T12:18:12.640041+0000","last_became_peered":"2024-05-04T12:18:12.640041+0000","last_unstale":"2024-05-04T12:18:24.296796+0000","last_undegraded":"2024-05-04T12:18:24.296796+0000","last_fullsized":"2024-05-04T12:18:24.296796+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_clean_scrub_stamp":"2024-05-04T12:18:11.260157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:15:02.089387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836180+0000","last_change":"2024-05-04T12:18:21.837361+0000","last_active":"2024-05-04T12:18:23.836180+0000","last_peered":"2024-05-04T12:18:23.836180+0000","last_clean":"2024-05-04T12:18:23.836180+0000","last_became_active":"2024-05-04T12:18:21.836437+0000","last_became_peered":"2024-05-04T12:18:21.836437+0000","last_unstale":"2024-05-04T12:18:23.836180+0000","last_undegraded":"2024-05-04T12:18:23.836180+0000","last_fullsized":"2024-05-04T12:18:23.836180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:48:12.147734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779602+0000","last_change":"2024-05-04T12:18:22.868591+0000","last_active":"2024-05-04T12:18:24.779602+0000","last_peered":"2024-05-04T12:18:24.779602+0000","last_clean":"2024-05-04T12:18:24.779602+0000","last_became_active":"2024-05-04T12:18:22.868158+0000","last_became_peered":"2024-05-04T12:18:22.868158+0000","last_unstale":"2024-05-04T12:18:24.779602+0000","last_undegraded":"2024-05-04T12:18:24.779602+0000","last_fullsized":"2024-05-04T12:18:24.779602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:05:46.918761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296711+0000","last_change":"2024-05-04T12:18:21.857252+0000","last_active":"2024-05-04T12:18:24.296711+0000","last_peered":"2024-05-04T12:18:24.296711+0000","last_clean":"2024-05-04T12:18:24.296711+0000","last_became_active":"2024-05-04T12:18:21.855446+0000","last_became_peered":"2024-05-04T12:18:21.855446+0000","last_unstale":"2024-05-04T12:18:24.296711+0000","last_undegraded":"2024-05-04T12:18:24.296711+0000","last_fullsized":"2024-05-04T12:18:24.296711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.246868+0000","last_change":"2024-05-04T12:18:22.862302+0000","last_active":"2024-05-04T12:18:24.246868+0000","last_peered":"2024-05-04T12:18:24.246868+0000","last_clean":"2024-05-04T12:18:24.246868+0000","last_became_active":"2024-05-04T12:18:22.859627+0000","last_became_peered":"2024-05-04T12:18:22.859627+0000","last_unstale":"2024-05-04T12:18:24.246868+0000","last_undegraded":"2024-05-04T12:18:24.246868+0000","last_fullsized":"2024-05-04T12:18:24.246868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:16:07.430915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.839684+0000","last_change":"2024-05-04T12:18:21.832787+0000","last_active":"2024-05-04T12:18:22.839684+0000","last_peered":"2024-05-04T12:18:22.839684+0000","last_clean":"2024-05-04T12:18:22.839684+0000","last_became_active":"2024-05-04T12:18:21.832247+0000","last_became_peered":"2024-05-04T12:18:21.832247+0000","last_unstale":"2024-05-04T12:18:22.839684+0000","last_undegraded":"2024-05-04T12:18:22.839684+0000","last_fullsized":"2024-05-04T12:18:22.839684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:48:12.147734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831403+0000","last_change":"2024-05-04T12:18:22.863972+0000","last_active":"2024-05-04T12:18:23.831403+0000","last_peered":"2024-05-04T12:18:23.831403+0000","last_clean":"2024-05-04T12:18:23.831403+0000","last_became_active":"2024-05-04T12:18:22.863122+0000","last_became_peered":"2024-05-04T12:18:22.863122+0000","last_unstale":"2024-05-04T12:18:23.831403+0000","last_undegraded":"2024-05-04T12:18:23.831403+0000","last_fullsized":"2024-05-04T12:18:23.831403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:47:19.879004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836266+0000","last_change":"2024-05-04T12:18:21.835740+0000","last_active":"2024-05-04T12:18:23.836266+0000","last_peered":"2024-05-04T12:18:23.836266+0000","last_clean":"2024-05-04T12:18:23.836266+0000","last_became_active":"2024-05-04T12:18:21.834835+0000","last_became_peered":"2024-05-04T12:18:21.834835+0000","last_unstale":"2024-05-04T12:18:23.836266+0000","last_undegraded":"2024-05-04T12:18:23.836266+0000","last_fullsized":"2024-05-04T12:18:23.836266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835634+0000","last_change":"2024-05-04T12:18:22.856686+0000","last_active":"2024-05-04T12:18:23.835634+0000","last_peered":"2024-05-04T12:18:23.835634+0000","last_clean":"2024-05-04T12:18:23.835634+0000","last_became_active":"2024-05-04T12:18:22.855883+0000","last_became_peered":"2024-05-04T12:18:22.855883+0000","last_unstale":"2024-05-04T12:18:23.835634+0000","last_undegraded":"2024-05-04T12:18:23.835634+0000","last_fullsized":"2024-05-04T12:18:23.835634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:19.851400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831728+0000","last_change":"2024-05-04T12:18:21.831578+0000","last_active":"2024-05-04T12:18:23.831728+0000","last_peered":"2024-05-04T12:18:23.831728+0000","last_clean":"2024-05-04T12:18:23.831728+0000","last_became_active":"2024-05-04T12:18:21.830473+0000","last_became_peered":"2024-05-04T12:18:21.830473+0000","last_unstale":"2024-05-04T12:18:23.831728+0000","last_undegraded":"2024-05-04T12:18:23.831728+0000","last_fullsized":"2024-05-04T12:18:23.831728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.837084+0000","last_change":"2024-05-04T12:18:22.878341+0000","last_active":"2024-05-04T12:18:23.837084+0000","last_peered":"2024-05-04T12:18:23.837084+0000","last_clean":"2024-05-04T12:18:23.837084+0000","last_became_active":"2024-05-04T12:18:22.878015+0000","last_became_peered":"2024-05-04T12:18:22.878015+0000","last_unstale":"2024-05-04T12:18:23.837084+0000","last_undegraded":"2024-05-04T12:18:23.837084+0000","last_fullsized":"2024-05-04T12:18:23.837084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:00:25.310837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298025+0000","last_change":"2024-05-04T12:18:21.857646+0000","last_active":"2024-05-04T12:18:24.298025+0000","last_peered":"2024-05-04T12:18:24.298025+0000","last_clean":"2024-05-04T12:18:24.298025+0000","last_became_active":"2024-05-04T12:18:21.856638+0000","last_became_peered":"2024-05-04T12:18:21.856638+0000","last_unstale":"2024-05-04T12:18:24.298025+0000","last_undegraded":"2024-05-04T12:18:24.298025+0000","last_fullsized":"2024-05-04T12:18:24.298025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:09:02.023381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831578+0000","last_change":"2024-05-04T12:18:22.862113+0000","last_active":"2024-05-04T12:18:23.831578+0000","last_peered":"2024-05-04T12:18:23.831578+0000","last_clean":"2024-05-04T12:18:23.831578+0000","last_became_active":"2024-05-04T12:18:22.861682+0000","last_became_peered":"2024-05-04T12:18:22.861682+0000","last_unstale":"2024-05-04T12:18:23.831578+0000","last_undegraded":"2024-05-04T12:18:23.831578+0000","last_fullsized":"2024-05-04T12:18:23.831578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.636826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.832118+0000","last_change":"2024-05-04T12:18:21.846869+0000","last_active":"2024-05-04T12:18:23.832118+0000","last_peered":"2024-05-04T12:18:23.832118+0000","last_clean":"2024-05-04T12:18:23.832118+0000","last_became_active":"2024-05-04T12:18:21.846514+0000","last_became_peered":"2024-05-04T12:18:21.846514+0000","last_unstale":"2024-05-04T12:18:23.832118+0000","last_undegraded":"2024-05-04T12:18:23.832118+0000","last_fullsized":"2024-05-04T12:18:23.832118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779211+0000","last_change":"2024-05-04T12:18:22.879784+0000","last_active":"2024-05-04T12:18:24.779211+0000","last_peered":"2024-05-04T12:18:24.779211+0000","last_clean":"2024-05-04T12:18:24.779211+0000","last_became_active":"2024-05-04T12:18:22.877733+0000","last_became_peered":"2024-05-04T12:18:22.877733+0000","last_unstale":"2024-05-04T12:18:24.779211+0000","last_undegraded":"2024-05-04T12:18:24.779211+0000","last_fullsized":"2024-05-04T12:18:24.779211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:10:47.797595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.863213+0000","last_change":"2024-05-04T12:18:21.854368+0000","last_active":"2024-05-04T12:18:22.863213+0000","last_peered":"2024-05-04T12:18:22.863213+0000","last_clean":"2024-05-04T12:18:22.863213+0000","last_became_active":"2024-05-04T12:18:21.852721+0000","last_became_peered":"2024-05-04T12:18:21.852721+0000","last_unstale":"2024-05-04T12:18:22.863213+0000","last_undegraded":"2024-05-04T12:18:22.863213+0000","last_fullsized":"2024-05-04T12:18:22.863213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247454+0000","last_change":"2024-05-04T12:18:22.853292+0000","last_active":"2024-05-04T12:18:24.247454+0000","last_peered":"2024-05-04T12:18:24.247454+0000","last_clean":"2024-05-04T12:18:24.247454+0000","last_became_active":"2024-05-04T12:18:22.852401+0000","last_became_peered":"2024-05-04T12:18:22.852401+0000","last_unstale":"2024-05-04T12:18:24.247454+0000","last_undegraded":"2024-05-04T12:18:24.247454+0000","last_fullsized":"2024-05-04T12:18:24.247454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296652+0000","last_change":"2024-05-04T12:18:21.858135+0000","last_active":"2024-05-04T12:18:24.296652+0000","last_peered":"2024-05-04T12:18:24.296652+0000","last_clean":"2024-05-04T12:18:24.296652+0000","last_became_active":"2024-05-04T12:18:21.856430+0000","last_became_peered":"2024-05-04T12:18:21.856430+0000","last_unstale":"2024-05-04T12:18:24.296652+0000","last_undegraded":"2024-05-04T12:18:24.296652+0000","last_fullsized":"2024-05-04T12:18:24.296652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835773+0000","last_change":"2024-05-04T12:18:22.852562+0000","last_active":"2024-05-04T12:18:23.835773+0000","last_peered":"2024-05-04T12:18:23.835773+0000","last_clean":"2024-05-04T12:18:23.835773+0000","last_became_active":"2024-05-04T12:18:22.851980+0000","last_became_peered":"2024-05-04T12:18:22.851980+0000","last_unstale":"2024-05-04T12:18:23.835773+0000","last_undegraded":"2024-05-04T12:18:23.835773+0000","last_fullsized":"2024-05-04T12:18:23.835773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:23.487598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831190+0000","last_change":"2024-05-04T12:18:21.837963+0000","last_active":"2024-05-04T12:18:23.831190+0000","last_peered":"2024-05-04T12:18:23.831190+0000","last_clean":"2024-05-04T12:18:23.831190+0000","last_became_active":"2024-05-04T12:18:21.837563+0000","last_became_peered":"2024-05-04T12:18:21.837563+0000","last_unstale":"2024-05-04T12:18:23.831190+0000","last_undegraded":"2024-05-04T12:18:23.831190+0000","last_fullsized":"2024-05-04T12:18:23.831190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778827+0000","last_change":"2024-05-04T12:18:22.874277+0000","last_active":"2024-05-04T12:18:24.778827+0000","last_peered":"2024-05-04T12:18:24.778827+0000","last_clean":"2024-05-04T12:18:24.778827+0000","last_became_active":"2024-05-04T12:18:22.869298+0000","last_became_peered":"2024-05-04T12:18:22.869298+0000","last_unstale":"2024-05-04T12:18:24.778827+0000","last_undegraded":"2024-05-04T12:18:24.778827+0000","last_fullsized":"2024-05-04T12:18:24.778827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831728+0000","last_change":"2024-05-04T12:18:21.831177+0000","last_active":"2024-05-04T12:18:23.831728+0000","last_peered":"2024-05-04T12:18:23.831728+0000","last_clean":"2024-05-04T12:18:23.831728+0000","last_became_active":"2024-05-04T12:18:21.830570+0000","last_became_peered":"2024-05-04T12:18:21.830570+0000","last_unstale":"2024-05-04T12:18:23.831728+0000","last_undegraded":"2024-05-04T12:18:23.831728+0000","last_fullsized":"2024-05-04T12:18:23.831728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779529+0000","last_change":"2024-05-04T12:18:22.880190+0000","last_active":"2024-05-04T12:18:24.779529+0000","last_peered":"2024-05-04T12:18:24.779529+0000","last_clean":"2024-05-04T12:18:24.779529+0000","last_became_active":"2024-05-04T12:18:22.879801+0000","last_became_peered":"2024-05-04T12:18:22.879801+0000","last_unstale":"2024-05-04T12:18:24.779529+0000","last_undegraded":"2024-05-04T12:18:24.779529+0000","last_fullsized":"2024-05-04T12:18:24.779529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:40:13.207623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297558+0000","last_change":"2024-05-04T12:18:21.860830+0000","last_active":"2024-05-04T12:18:24.297558+0000","last_peered":"2024-05-04T12:18:24.297558+0000","last_clean":"2024-05-04T12:18:24.297558+0000","last_became_active":"2024-05-04T12:18:21.860156+0000","last_became_peered":"2024-05-04T12:18:21.860156+0000","last_unstale":"2024-05-04T12:18:24.297558+0000","last_undegraded":"2024-05-04T12:18:24.297558+0000","last_fullsized":"2024-05-04T12:18:24.297558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:39.485391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-04T12:18:22.864508+0000","last_change":"2024-05-04T12:18:22.860205+0000","last_active":"2024-05-04T12:18:22.864508+0000","last_peered":"2024-05-04T12:18:22.864508+0000","last_clean":"2024-05-04T12:18:22.864508+0000","last_became_active":"2024-05-04T12:18:22.859825+0000","last_became_peered":"2024-05-04T12:18:22.859825+0000","last_unstale":"2024-05-04T12:18:22.864508+0000","last_undegraded":"2024-05-04T12:18:22.864508+0000","last_fullsized":"2024-05-04T12:18:22.864508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:03:36.140108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.837078+0000","last_change":"2024-05-04T12:18:22.844339+0000","last_active":"2024-05-04T12:18:23.837078+0000","last_peered":"2024-05-04T12:18:23.837078+0000","last_clean":"2024-05-04T12:18:23.837078+0000","last_became_active":"2024-05-04T12:18:22.843807+0000","last_became_peered":"2024-05-04T12:18:22.843807+0000","last_unstale":"2024-05-04T12:18:23.837078+0000","last_undegraded":"2024-05-04T12:18:23.837078+0000","last_fullsized":"2024-05-04T12:18:23.837078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:16:56.544684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297584+0000","last_change":"2024-05-04T12:18:21.860670+0000","last_active":"2024-05-04T12:18:24.297584+0000","last_peered":"2024-05-04T12:18:24.297584+0000","last_clean":"2024-05-04T12:18:24.297584+0000","last_became_active":"2024-05-04T12:18:21.859826+0000","last_became_peered":"2024-05-04T12:18:21.859826+0000","last_unstale":"2024-05-04T12:18:24.297584+0000","last_undegraded":"2024-05-04T12:18:24.297584+0000","last_fullsized":"2024-05-04T12:18:24.297584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836737+0000","last_change":"2024-05-04T12:18:22.846218+0000","last_active":"2024-05-04T12:18:23.836737+0000","last_peered":"2024-05-04T12:18:23.836737+0000","last_clean":"2024-05-04T12:18:23.836737+0000","last_became_active":"2024-05-04T12:18:22.845967+0000","last_became_peered":"2024-05-04T12:18:22.845967+0000","last_unstale":"2024-05-04T12:18:23.836737+0000","last_undegraded":"2024-05-04T12:18:23.836737+0000","last_fullsized":"2024-05-04T12:18:23.836737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:05:46.918761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297709+0000","last_change":"2024-05-04T12:18:21.860069+0000","last_active":"2024-05-04T12:18:24.297709+0000","last_peered":"2024-05-04T12:18:24.297709+0000","last_clean":"2024-05-04T12:18:24.297709+0000","last_became_active":"2024-05-04T12:18:21.859583+0000","last_became_peered":"2024-05-04T12:18:21.859583+0000","last_unstale":"2024-05-04T12:18:24.297709+0000","last_undegraded":"2024-05-04T12:18:24.297709+0000","last_fullsized":"2024-05-04T12:18:24.297709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:03:35.131163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247407+0000","last_change":"2024-05-04T12:18:22.856277+0000","last_active":"2024-05-04T12:18:24.247407+0000","last_peered":"2024-05-04T12:18:24.247407+0000","last_clean":"2024-05-04T12:18:24.247407+0000","last_became_active":"2024-05-04T12:18:22.855958+0000","last_became_peered":"2024-05-04T12:18:22.855958+0000","last_unstale":"2024-05-04T12:18:24.247407+0000","last_undegraded":"2024-05-04T12:18:24.247407+0000","last_fullsized":"2024-05-04T12:18:24.247407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.636826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779181+0000","last_change":"2024-05-04T12:18:21.835661+0000","last_active":"2024-05-04T12:18:24.779181+0000","last_peered":"2024-05-04T12:18:24.779181+0000","last_clean":"2024-05-04T12:18:24.779181+0000","last_became_active":"2024-05-04T12:18:21.835299+0000","last_became_peered":"2024-05-04T12:18:21.835299+0000","last_unstale":"2024-05-04T12:18:24.779181+0000","last_undegraded":"2024-05-04T12:18:24.779181+0000","last_fullsized":"2024-05-04T12:18:24.779181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:09:02.023381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779181+0000","last_change":"2024-05-04T12:18:22.878092+0000","last_active":"2024-05-04T12:18:24.779181+0000","last_peered":"2024-05-04T12:18:24.779181+0000","last_clean":"2024-05-04T12:18:24.779181+0000","last_became_active":"2024-05-04T12:18:22.875650+0000","last_became_peered":"2024-05-04T12:18:22.875650+0000","last_unstale":"2024-05-04T12:18:24.779181+0000","last_undegraded":"2024-05-04T12:18:24.779181+0000","last_fullsized":"2024-05-04T12:18:24.779181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247663+0000","last_change":"2024-05-04T12:18:21.847981+0000","last_active":"2024-05-04T12:18:24.247663+0000","last_peered":"2024-05-04T12:18:24.247663+0000","last_clean":"2024-05-04T12:18:24.247663+0000","last_became_active":"2024-05-04T12:18:21.847556+0000","last_became_peered":"2024-05-04T12:18:21.847556+0000","last_unstale":"2024-05-04T12:18:24.247663+0000","last_undegraded":"2024-05-04T12:18:24.247663+0000","last_fullsized":"2024-05-04T12:18:24.247663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:07:48.655726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":70917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":70917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":69305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T12:18:27.303 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-04T12:18:27.304 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-04T12:18:27.304 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-04T12:18:27.304 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-04T12:18:27.304 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-04T12:18:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b476c640 1 Processor -- start 2024-05-04T12:18:27.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b476c640 1 -- start start 2024-05-04T12:18:27.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b476c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7ac15c450 0x7fc7ac0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b476c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc7ac0ccf10 0x7fc7ac0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b476c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc7ac0d28c0 0x7fc7ac06ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b476c640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc7ac0cec20 con 0x7fc7ac0ccf10 2024-05-04T12:18:27.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b476c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fc7ac0cfec0 con 0x7fc7ac0d28c0 2024-05-04T12:18:27.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b476c640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ac160af0 con 0x7fc7ac15c450 2024-05-04T12:18:27.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b1d07640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc7ac0ccf10 0x7fc7ac0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b1d07640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc7ac0ccf10 0x7fc7ac0d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53312/0 (socket says 172.21.15.52:53312) 2024-05-04T12:18:27.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b1d07640 1 -- 172.21.15.52:0/2647777157 learned_addr learned my addr 172.21.15.52:0/2647777157 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:27.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b2d09640 1 --2- 172.21.15.52:0/2647777157 >> v2:172.21.15.111:3300/0 conn(0x7fc7ac0d28c0 0x7fc7ac06ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b2508640 1 --2- 172.21.15.52:0/2647777157 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7ac15c450 0x7fc7ac0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b1d07640 1 -- 172.21.15.52:0/2647777157 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7ac15c450 msgr2=0x7fc7ac0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:27.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b1d07640 1 --2- 172.21.15.52:0/2647777157 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7ac15c450 0x7fc7ac0ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b1d07640 1 -- 172.21.15.52:0/2647777157 >> v2:172.21.15.111:3300/0 conn(0x7fc7ac0d28c0 msgr2=0x7fc7ac06ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:27.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b1d07640 1 --2- 172.21.15.52:0/2647777157 >> v2:172.21.15.111:3300/0 conn(0x7fc7ac0d28c0 0x7fc7ac06ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.470+0000 7fc7b1d07640 1 -- 172.21.15.52:0/2647777157 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac160dd0 con 0x7fc7ac0ccf10 2024-05-04T12:18:27.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.474+0000 7fc7b1d07640 1 --2- 172.21.15.52:0/2647777157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7ac0ccf10 0x7fc7ac0d2380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac06e860 tx=0x7fc79c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e608b25ef1e74dc server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:27.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.474+0000 7fc7b1506640 1 -- 172.21.15.52:0/2647777157 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0b0070 con 0x7fc7ac0ccf10 2024-05-04T12:18:27.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.474+0000 7fc7b1506640 1 -- 172.21.15.52:0/2647777157 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79c0ab600 con 0x7fc7ac0ccf10 2024-05-04T12:18:27.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.474+0000 7fc7b1506640 1 -- 172.21.15.52:0/2647777157 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0ab9d0 con 0x7fc7ac0ccf10 2024-05-04T12:18:27.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.474+0000 7fc7b476c640 1 -- 172.21.15.52:0/2647777157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7ac0ccf10 msgr2=0x7fc7ac0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:27.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.474+0000 7fc7b476c640 1 --2- 172.21.15.52:0/2647777157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7ac0ccf10 0x7fc7ac0d2380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac06e860 tx=0x7fc79c01c260 comp rx=0 tx=0).stop 2024-05-04T12:18:27.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.474+0000 7fc7b476c640 1 -- 172.21.15.52:0/2647777157 shutdown_connections 2024-05-04T12:18:27.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.474+0000 7fc7b476c640 1 --2- 172.21.15.52:0/2647777157 >> v2:172.21.15.111:3300/0 conn(0x7fc7ac0d28c0 0x7fc7ac06ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.474+0000 7fc7b476c640 1 --2- 172.21.15.52:0/2647777157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7ac0ccf10 0x7fc7ac0d2380 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.474+0000 7fc7b476c640 1 --2- 172.21.15.52:0/2647777157 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7ac15c450 0x7fc7ac0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.474+0000 7fc7b476c640 1 -- 172.21.15.52:0/2647777157 >> 172.21.15.52:0/2647777157 conn(0x7fc7ac06b210 msgr2=0x7fc7ac06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:27.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.474+0000 7fc7b476c640 1 -- 172.21.15.52:0/2647777157 shutdown_connections 2024-05-04T12:18:27.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.474+0000 7fc7b476c640 1 -- 172.21.15.52:0/2647777157 wait complete. 2024-05-04T12:18:27.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b476c640 1 Processor -- start 2024-05-04T12:18:27.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b476c640 1 -- start start 2024-05-04T12:18:27.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b476c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7ac0d28c0 0x7fc7ac149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b476c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7ac15c450 0x7fc7ac152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b476c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7ac14a490 0x7fc7ac1528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b476c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac15c9b0 con 0x7fc7ac15c450 2024-05-04T12:18:27.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b476c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac15c830 con 0x7fc7ac14a490 2024-05-04T12:18:27.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b476c640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ac0d02f0 con 0x7fc7ac0d28c0 2024-05-04T12:18:27.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b1d07640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7ac15c450 0x7fc7ac152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b1d07640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7ac15c450 0x7fc7ac152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53320/0 (socket says 172.21.15.52:53320) 2024-05-04T12:18:27.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b1d07640 1 -- 172.21.15.52:0/3229762867 learned_addr learned my addr 172.21.15.52:0/3229762867 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:27.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b2508640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7ac0d28c0 0x7fc7ac149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b2d09640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7ac14a490 0x7fc7ac1528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b1d07640 1 -- 172.21.15.52:0/3229762867 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7ac0d28c0 msgr2=0x7fc7ac149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:27.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b1d07640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7ac0d28c0 0x7fc7ac149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b1d07640 1 -- 172.21.15.52:0/3229762867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7ac14a490 msgr2=0x7fc7ac1528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:27.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b1d07640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7ac14a490 0x7fc7ac1528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b1d07640 1 -- 172.21.15.52:0/3229762867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac1515d0 con 0x7fc7ac15c450 2024-05-04T12:18:27.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b2d09640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7ac14a490 0x7fc7ac1528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:27.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b1d07640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7ac15c450 0x7fc7ac152370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0ae040 tx=0x7fc79c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:27.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/3229762867 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0b0040 con 0x7fc7ac15c450 2024-05-04T12:18:27.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/3229762867 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79c0b9070 con 0x7fc7ac15c450 2024-05-04T12:18:27.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/3229762867 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0b4680 con 0x7fc7ac15c450 2024-05-04T12:18:27.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b476c640 1 -- 172.21.15.52:0/3229762867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ac1518b0 con 0x7fc7ac15c450 2024-05-04T12:18:27.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.478+0000 7fc7b476c640 1 -- 172.21.15.52:0/3229762867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ac151e70 con 0x7fc7ac15c450 2024-05-04T12:18:27.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.482+0000 7fc7b476c640 1 -- 172.21.15.52:0/3229762867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ac062e50 con 0x7fc7ac15c450 2024-05-04T12:18:27.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.482+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/3229762867 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc79c0b4820 con 0x7fc7ac15c450 2024-05-04T12:18:27.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.482+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc7900678c0 0x7fc790069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.482+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/3229762867 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc79c11f880 con 0x7fc7ac15c450 2024-05-04T12:18:27.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.482+0000 7fc7b2508640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc7900678c0 0x7fc790069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.482+0000 7fc7b2508640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc7900678c0 0x7fc790069d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009ac0 tx=0x7fc7a8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:27.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.486+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/3229762867 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc79c016110 con 0x7fc7ac15c450 2024-05-04T12:18:27.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.686+0000 7fc7b476c640 1 -- 172.21.15.52:0/3229762867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc7ac075320 con 0x7fc7ac15c450 2024-05-04T12:18:27.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.694+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/3229762867 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+166882 (secure 0 0 0) 0x7fc79c0f03c0 con 0x7fc7ac15c450 2024-05-04T12:18:27.699 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:27.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.698+0000 7fc7b476c640 1 -- 172.21.15.52:0/3229762867 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc7900678c0 msgr2=0x7fc790069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:27.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.698+0000 7fc7b476c640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc7900678c0 0x7fc790069d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009ac0 tx=0x7fc7a8009190 comp rx=0 tx=0).stop 2024-05-04T12:18:27.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.698+0000 7fc7b476c640 1 -- 172.21.15.52:0/3229762867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7ac15c450 msgr2=0x7fc7ac152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:27.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.698+0000 7fc7b476c640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7ac15c450 0x7fc7ac152370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0ae040 tx=0x7fc79c019000 comp rx=0 tx=0).stop 2024-05-04T12:18:27.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.698+0000 7fc7b476c640 1 -- 172.21.15.52:0/3229762867 shutdown_connections 2024-05-04T12:18:27.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.698+0000 7fc7b476c640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc7900678c0 0x7fc790069d80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.698+0000 7fc7b476c640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7ac14a490 0x7fc7ac1528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.698+0000 7fc7b476c640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7ac15c450 0x7fc7ac152370 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.698+0000 7fc7b476c640 1 --2- 172.21.15.52:0/3229762867 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7ac0d28c0 0x7fc7ac149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.698+0000 7fc7b476c640 1 -- 172.21.15.52:0/3229762867 >> 172.21.15.52:0/3229762867 conn(0x7fc7ac06b210 msgr2=0x7fc7ac14c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:27.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.698+0000 7fc7b476c640 1 -- 172.21.15.52:0/3229762867 shutdown_connections 2024-05-04T12:18:27.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.698+0000 7fc7b476c640 1 -- 172.21.15.52:0/3229762867 wait complete. 2024-05-04T12:18:27.718 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":405834661},{"type":"v1","addr":"172.21.15.52:6833","nonce":405834661}]},"active_addr":"172.21.15.52:6833/405834661","active_change":"2024-05-04T12:18:02.726732+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":1161975538}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":3438767425}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":434321319}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":172805147}]}]} 2024-05-04T12:18:27.721 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-04T12:18:27.721 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-04T12:18:27.721 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:27.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb37cb37640 1 Processor -- start 2024-05-04T12:18:27.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb37cb37640 1 -- start start 2024-05-04T12:18:27.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb37cb37640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb37806f560 0x7fb3780ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb37cb37640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3780cfb60 0x7fb3780d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb37cb37640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb378146540 con 0x7fb378146160 2024-05-04T12:18:27.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb37cb37640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb3780cd140 con 0x7fb3780cfb60 2024-05-04T12:18:27.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb37cb37640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fb3780d0e10 con 0x7fb37806f560 2024-05-04T12:18:27.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb376d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3780cfb60 0x7fb3780d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb376d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3780cfb60 0x7fb3780d18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53330/0 (socket says 172.21.15.52:53330) 2024-05-04T12:18:27.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb376d9d640 1 -- 172.21.15.52:0/1566996531 learned_addr learned my addr 172.21.15.52:0/1566996531 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:27.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb375d9b640 1 --2- 172.21.15.52:0/1566996531 >> v2:172.21.15.111:3300/0 conn(0x7fb37806f560 0x7fb3780ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb376d9d640 1 -- 172.21.15.52:0/1566996531 >> v2:172.21.15.111:3300/0 conn(0x7fb37806f560 msgr2=0x7fb3780ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:27.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb376d9d640 1 --2- 172.21.15.52:0/1566996531 >> v2:172.21.15.111:3300/0 conn(0x7fb37806f560 0x7fb3780ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb376d9d640 1 -- 172.21.15.52:0/1566996531 >> v1:172.21.15.111:6789/0 conn(0x7fb378146160 legacy=0x7fb37806ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:27.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.838+0000 7fb376d9d640 1 -- 172.21.15.52:0/1566996531 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3780d10f0 con 0x7fb3780cfb60 2024-05-04T12:18:27.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb376d9d640 1 --2- 172.21.15.52:0/1566996531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3780cfb60 0x7fb3780d18f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb3780b1590 tx=0x7fb37001c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05ed9b344d52e9d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:27.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37559a640 1 -- 172.21.15.52:0/1566996531 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3700a5540 con 0x7fb3780cfb60 2024-05-04T12:18:27.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37559a640 1 -- 172.21.15.52:0/1566996531 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3700a56e0 con 0x7fb3780cfb60 2024-05-04T12:18:27.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37559a640 1 -- 172.21.15.52:0/1566996531 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3700a5ab0 con 0x7fb3780cfb60 2024-05-04T12:18:27.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 -- 172.21.15.52:0/1566996531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3780cfb60 msgr2=0x7fb3780d18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:27.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 --2- 172.21.15.52:0/1566996531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3780cfb60 0x7fb3780d18f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb3780b1590 tx=0x7fb37001c7d0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 -- 172.21.15.52:0/1566996531 shutdown_connections 2024-05-04T12:18:27.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 --2- 172.21.15.52:0/1566996531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3780cfb60 0x7fb3780d18f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 --2- 172.21.15.52:0/1566996531 >> v2:172.21.15.111:3300/0 conn(0x7fb37806f560 0x7fb3780ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 -- 172.21.15.52:0/1566996531 >> 172.21.15.52:0/1566996531 conn(0x7fb37806b210 msgr2=0x7fb37806b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:27.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 -- 172.21.15.52:0/1566996531 shutdown_connections 2024-05-04T12:18:27.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 -- 172.21.15.52:0/1566996531 wait complete. 2024-05-04T12:18:27.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 Processor -- start 2024-05-04T12:18:27.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 -- start start 2024-05-04T12:18:27.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb37806f560 0x7fb378157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3780cfb60 0x7fb378155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb378146160 0x7fb378166330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780d27c0 con 0x7fb378146160 2024-05-04T12:18:27.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780d2640 con 0x7fb37806f560 2024-05-04T12:18:27.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb37cb37640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fb3780d2940 con 0x7fb3780cfb60 2024-05-04T12:18:27.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.842+0000 7fb376d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb378146160 0x7fb378166330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb376d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb378146160 0x7fb378166330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53346/0 (socket says 172.21.15.52:53346) 2024-05-04T12:18:27.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb376d9d640 1 -- 172.21.15.52:0/2948700845 learned_addr learned my addr 172.21.15.52:0/2948700845 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:27.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb37659c640 1 --2- 172.21.15.52:0/2948700845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb37806f560 0x7fb378157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb375d9b640 1 --2- 172.21.15.52:0/2948700845 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3780cfb60 0x7fb378155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb376d9d640 1 -- 172.21.15.52:0/2948700845 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3780cfb60 msgr2=0x7fb378155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:27.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb376d9d640 1 --2- 172.21.15.52:0/2948700845 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3780cfb60 0x7fb378155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb376d9d640 1 -- 172.21.15.52:0/2948700845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb37806f560 msgr2=0x7fb378157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:27.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb376d9d640 1 --2- 172.21.15.52:0/2948700845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb37806f560 0x7fb378157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:27.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb376d9d640 1 -- 172.21.15.52:0/2948700845 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb378156230 con 0x7fb378146160 2024-05-04T12:18:27.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb376d9d640 1 --2- 172.21.15.52:0/2948700845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb378146160 0x7fb378166330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb3700af040 tx=0x7fb370019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:27.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb366ffd640 1 -- 172.21.15.52:0/2948700845 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370009510 con 0x7fb378146160 2024-05-04T12:18:27.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb366ffd640 1 -- 172.21.15.52:0/2948700845 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3700be070 con 0x7fb378146160 2024-05-04T12:18:27.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb37cb37640 1 -- 172.21.15.52:0/2948700845 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb378156510 con 0x7fb378146160 2024-05-04T12:18:27.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb366ffd640 1 -- 172.21.15.52:0/2948700845 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3700b88d0 con 0x7fb378146160 2024-05-04T12:18:27.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb37cb37640 1 -- 172.21.15.52:0/2948700845 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb378156ad0 con 0x7fb378146160 2024-05-04T12:18:27.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb37cb37640 1 -- 172.21.15.52:0/2948700845 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb378062e50 con 0x7fb378146160 2024-05-04T12:18:27.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb366ffd640 1 -- 172.21.15.52:0/2948700845 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb3700b8a70 con 0x7fb378146160 2024-05-04T12:18:27.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb366ffd640 1 --2- 172.21.15.52:0/2948700845 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3580678c0 0x7fb358069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:27.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb366ffd640 1 -- 172.21.15.52:0/2948700845 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb370120cb0 con 0x7fb378146160 2024-05-04T12:18:27.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb37659c640 1 --2- 172.21.15.52:0/2948700845 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3580678c0 0x7fb358069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:27.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.846+0000 7fb37659c640 1 --2- 172.21.15.52:0/2948700845 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3580678c0 0x7fb358069d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb3780cc960 tx=0x7fb368009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:27.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:27.850+0000 7fb366ffd640 1 -- 172.21.15.52:0/2948700845 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb378146540 con 0x7fb378146160 2024-05-04T12:18:28.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.014+0000 7fb37cb37640 1 -- 172.21.15.52:0/2948700845 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb378075320 con 0x7fb378146160 2024-05-04T12:18:28.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.014+0000 7fb366ffd640 1 -- 172.21.15.52:0/2948700845 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fb3700f1910 con 0x7fb378146160 2024-05-04T12:18:28.018 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:28.018 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:28.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.018+0000 7fb37cb37640 1 -- 172.21.15.52:0/2948700845 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3580678c0 msgr2=0x7fb358069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.018+0000 7fb37cb37640 1 --2- 172.21.15.52:0/2948700845 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3580678c0 0x7fb358069d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb3780cc960 tx=0x7fb368009060 comp rx=0 tx=0).stop 2024-05-04T12:18:28.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.018+0000 7fb37cb37640 1 -- 172.21.15.52:0/2948700845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb378146160 msgr2=0x7fb378166330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.018+0000 7fb37cb37640 1 --2- 172.21.15.52:0/2948700845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb378146160 0x7fb378166330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb3700af040 tx=0x7fb370019570 comp rx=0 tx=0).stop 2024-05-04T12:18:28.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.018+0000 7fb37cb37640 1 -- 172.21.15.52:0/2948700845 shutdown_connections 2024-05-04T12:18:28.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.018+0000 7fb37cb37640 1 --2- 172.21.15.52:0/2948700845 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3580678c0 0x7fb358069d80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.018+0000 7fb37cb37640 1 --2- 172.21.15.52:0/2948700845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb378146160 0x7fb378166330 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.018+0000 7fb37cb37640 1 --2- 172.21.15.52:0/2948700845 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3780cfb60 0x7fb378155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.018+0000 7fb37cb37640 1 --2- 172.21.15.52:0/2948700845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb37806f560 0x7fb378157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.018+0000 7fb37cb37640 1 -- 172.21.15.52:0/2948700845 >> 172.21.15.52:0/2948700845 conn(0x7fb37806b210 msgr2=0x7fb378145380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.018+0000 7fb37cb37640 1 -- 172.21.15.52:0/2948700845 shutdown_connections 2024-05-04T12:18:28.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.018+0000 7fb37cb37640 1 -- 172.21.15.52:0/2948700845 wait complete. 2024-05-04T12:18:28.036 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-04T12:18:28.036 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:28.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a313d2640 1 Processor -- start 2024-05-04T12:18:28.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a313d2640 1 -- start start 2024-05-04T12:18:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a313d2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2a2c0ccdb0 0x7f2a2c0723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a313d2640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a2c072aa0 con 0x7f2a2c0cd190 2024-05-04T12:18:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a313d2640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a2c0708e0 con 0x7f2a2c15c7f0 2024-05-04T12:18:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a313d2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f2a2c070690 con 0x7f2a2c0ccdb0 2024-05-04T12:18:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2b7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f2a2c0cd190 0x7f2a2c06de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51486/0 (socket says 172.21.15.52:51486) 2024-05-04T12:18:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2b7fe640 1 -- 172.21.15.52:0/547297906 learned_addr learned my addr 172.21.15.52:0/547297906 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2affd640 1 --2- 172.21.15.52:0/547297906 >> v2:172.21.15.111:3300/0 conn(0x7f2a2c0ccdb0 0x7f2a2c0723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 -- 172.21.15.52:0/547297906 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2164141673 0 0) 0x7f2a2c072aa0 con 0x7f2a2c0cd190 2024-05-04T12:18:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 -- 172.21.15.52:0/547297906 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a18003530 con 0x7f2a2c0cd190 2024-05-04T12:18:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 -- 172.21.15.52:0/547297906 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3701220408 0 0) 0x7f2a18003530 con 0x7f2a2c0cd190 2024-05-04T12:18:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 -- 172.21.15.52:0/547297906 >> v2:172.21.15.111:3300/0 conn(0x7f2a2c0ccdb0 msgr2=0x7f2a2c0723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 --2- 172.21.15.52:0/547297906 >> v2:172.21.15.111:3300/0 conn(0x7f2a2c0ccdb0 0x7f2a2c0723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 -- 172.21.15.52:0/547297906 >> v1:172.21.15.111:6789/0 conn(0x7f2a2c15c7f0 legacy=0x7f2a2c158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 -- 172.21.15.52:0/547297906 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2c070bc0 con 0x7f2a2c0cd190 2024-05-04T12:18:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 -- 172.21.15.52:0/547297906 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f2a24001a00 con 0x7f2a2c0cd190 2024-05-04T12:18:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 -- 172.21.15.52:0/547297906 >> v1:172.21.15.52:6789/0 conn(0x7f2a2c0cd190 legacy=0x7f2a2c06de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 --2- 172.21.15.52:0/547297906 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a180039c0 0x7f2a18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 --2- 172.21.15.52:0/547297906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a18004810 0x7f2a18004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 --2- 172.21.15.52:0/547297906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a180051d0 0x7f2a1800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 -- 172.21.15.52:0/547297906 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c070690 con 0x7f2a18004810 2024-05-04T12:18:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 -- 172.21.15.52:0/547297906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1800c170 con 0x7f2a180051d0 2024-05-04T12:18:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2a7fc640 1 -- 172.21.15.52:0/547297906 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f2a1800c370 con 0x7f2a180039c0 2024-05-04T12:18:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a23fff640 1 --2- 172.21.15.52:0/547297906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a18004810 0x7f2a18004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2affd640 1 --2- 172.21.15.52:0/547297906 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a180039c0 0x7f2a18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2b7fe640 1 --2- 172.21.15.52:0/547297906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a180051d0 0x7f2a1800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a23fff640 1 -- 172.21.15.52:0/547297906 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a180039c0 msgr2=0x7f2a18003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a23fff640 1 --2- 172.21.15.52:0/547297906 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a180039c0 0x7f2a18003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a23fff640 1 -- 172.21.15.52:0/547297906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a180051d0 msgr2=0x7f2a1800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a23fff640 1 --2- 172.21.15.52:0/547297906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a180051d0 0x7f2a1800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a2b7fe640 1 -- 172.21.15.52:0/547297906 reap_dead start 2024-05-04T12:18:28.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.154+0000 7f2a23fff640 1 -- 172.21.15.52:0/547297906 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a18003530 con 0x7f2a18004810 2024-05-04T12:18:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a23fff640 1 --2- 172.21.15.52:0/547297906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a18004810 0x7f2a18004c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2a14001780 tx=0x7f2a1401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7ef9c3ca83df97 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a2a7fc640 1 -- 172.21.15.52:0/547297906 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a140013a0 con 0x7f2a18004810 2024-05-04T12:18:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a2a7fc640 1 -- 172.21.15.52:0/547297906 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a140052f0 con 0x7f2a18004810 2024-05-04T12:18:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a2a7fc640 1 -- 172.21.15.52:0/547297906 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14003840 con 0x7f2a18004810 2024-05-04T12:18:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 -- 172.21.15.52:0/547297906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a18004810 msgr2=0x7f2a18004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 --2- 172.21.15.52:0/547297906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a18004810 0x7f2a18004c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2a14001780 tx=0x7f2a1401c400 comp rx=0 tx=0).stop 2024-05-04T12:18:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 -- 172.21.15.52:0/547297906 shutdown_connections 2024-05-04T12:18:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 --2- 172.21.15.52:0/547297906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a18004810 0x7f2a18004c90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 -- 172.21.15.52:0/547297906 >> 172.21.15.52:0/547297906 conn(0x7f2a2c0b14a0 msgr2=0x7f2a2c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 -- 172.21.15.52:0/547297906 shutdown_connections 2024-05-04T12:18:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 -- 172.21.15.52:0/547297906 wait complete. 2024-05-04T12:18:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 Processor -- start 2024-05-04T12:18:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 -- start start 2024-05-04T12:18:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a18004810 0x7f2a2c154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a2c155260 0x7f2a2c1f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a2c152f50 0x7f2a2c1f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c06de70 con 0x7f2a2c155260 2024-05-04T12:18:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c159060 con 0x7f2a18004810 2024-05-04T12:18:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.158+0000 7f2a313d2640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f2a2c06dff0 con 0x7f2a2c152f50 2024-05-04T12:18:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a23fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a2c155260 0x7f2a2c1f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a23fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a2c155260 0x7f2a2c1f55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53360/0 (socket says 172.21.15.52:53360) 2024-05-04T12:18:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a23fff640 1 -- 172.21.15.52:0/3960371492 learned_addr learned my addr 172.21.15.52:0/3960371492 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a2b7fe640 1 --2- 172.21.15.52:0/3960371492 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a2c152f50 0x7f2a2c1f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a2affd640 1 --2- 172.21.15.52:0/3960371492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a18004810 0x7f2a2c154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a23fff640 1 -- 172.21.15.52:0/3960371492 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a2c152f50 msgr2=0x7f2a2c1f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a23fff640 1 --2- 172.21.15.52:0/3960371492 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a2c152f50 0x7f2a2c1f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a23fff640 1 -- 172.21.15.52:0/3960371492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a18004810 msgr2=0x7f2a2c154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a23fff640 1 --2- 172.21.15.52:0/3960371492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a18004810 0x7f2a2c154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a23fff640 1 -- 172.21.15.52:0/3960371492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2c155750 con 0x7f2a2c155260 2024-05-04T12:18:28.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a23fff640 1 --2- 172.21.15.52:0/3960371492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a2c155260 0x7f2a2c1f55a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2a1401c230 tx=0x7f2a18005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a237fe640 1 -- 172.21.15.52:0/3960371492 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14007a40 con 0x7f2a2c155260 2024-05-04T12:18:28.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a237fe640 1 -- 172.21.15.52:0/3960371492 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a140ba070 con 0x7f2a2c155260 2024-05-04T12:18:28.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a313d2640 1 -- 172.21.15.52:0/3960371492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2c155a30 con 0x7f2a2c155260 2024-05-04T12:18:28.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a237fe640 1 -- 172.21.15.52:0/3960371492 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14007270 con 0x7f2a2c155260 2024-05-04T12:18:28.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a313d2640 1 -- 172.21.15.52:0/3960371492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a2c1f6800 con 0x7f2a2c155260 2024-05-04T12:18:28.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a313d2640 1 -- 172.21.15.52:0/3960371492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a2c062e50 con 0x7f2a2c155260 2024-05-04T12:18:28.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a237fe640 1 -- 172.21.15.52:0/3960371492 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2a14005280 con 0x7f2a2c155260 2024-05-04T12:18:28.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a237fe640 1 --2- 172.21.15.52:0/3960371492 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2a0c0678c0 0x7f2a0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a237fe640 1 -- 172.21.15.52:0/3960371492 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2a1411fdb0 con 0x7f2a2c155260 2024-05-04T12:18:28.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a2affd640 1 --2- 172.21.15.52:0/3960371492 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2a0c0678c0 0x7f2a0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.162+0000 7f2a2affd640 1 --2- 172.21.15.52:0/3960371492 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2a0c0678c0 0x7f2a0c069d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009ba0 tx=0x7f2a1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.166+0000 7f2a237fe640 1 -- 172.21.15.52:0/3960371492 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2a14016110 con 0x7f2a2c155260 2024-05-04T12:18:28.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.330+0000 7f2a313d2640 1 -- 172.21.15.52:0/3960371492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a2c075320 con 0x7f2a2c155260 2024-05-04T12:18:28.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.330+0000 7f2a237fe640 1 -- 172.21.15.52:0/3960371492 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f2a140f0a10 con 0x7f2a2c155260 2024-05-04T12:18:28.334 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:28.334 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:28.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.334+0000 7f2a313d2640 1 -- 172.21.15.52:0/3960371492 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2a0c0678c0 msgr2=0x7f2a0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.334+0000 7f2a313d2640 1 --2- 172.21.15.52:0/3960371492 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2a0c0678c0 0x7f2a0c069d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009ba0 tx=0x7f2a1c008040 comp rx=0 tx=0).stop 2024-05-04T12:18:28.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.334+0000 7f2a313d2640 1 -- 172.21.15.52:0/3960371492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a2c155260 msgr2=0x7f2a2c1f55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.334+0000 7f2a313d2640 1 --2- 172.21.15.52:0/3960371492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a2c155260 0x7f2a2c1f55a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2a1401c230 tx=0x7f2a18005840 comp rx=0 tx=0).stop 2024-05-04T12:18:28.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.334+0000 7f2a313d2640 1 -- 172.21.15.52:0/3960371492 shutdown_connections 2024-05-04T12:18:28.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.334+0000 7f2a313d2640 1 --2- 172.21.15.52:0/3960371492 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2a0c0678c0 0x7f2a0c069d80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.334+0000 7f2a313d2640 1 --2- 172.21.15.52:0/3960371492 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a2c152f50 0x7f2a2c1f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.334+0000 7f2a313d2640 1 --2- 172.21.15.52:0/3960371492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a2c155260 0x7f2a2c1f55a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.334+0000 7f2a313d2640 1 --2- 172.21.15.52:0/3960371492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a18004810 0x7f2a2c154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.334+0000 7f2a313d2640 1 -- 172.21.15.52:0/3960371492 >> 172.21.15.52:0/3960371492 conn(0x7f2a2c0b14a0 msgr2=0x7f2a2c15ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.334+0000 7f2a313d2640 1 -- 172.21.15.52:0/3960371492 shutdown_connections 2024-05-04T12:18:28.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.334+0000 7f2a313d2640 1 -- 172.21.15.52:0/3960371492 wait complete. 2024-05-04T12:18:28.354 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-04T12:18:28.355 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-04T12:18:28.355 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-04T12:18:28.355 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-04T12:18:28.356 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-04T12:18:28.356 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-04T12:18:28.357 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-04T12:18:28.357 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-04T12:18:28.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.534+0000 7fc8c559d640 1 Processor -- start 2024-05-04T12:18:28.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.534+0000 7fc8c559d640 1 -- start start 2024-05-04T12:18:28.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.534+0000 7f69e6f85640 1 Processor -- start 2024-05-04T12:18:28.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.534+0000 7f69e6f85640 1 -- start start 2024-05-04T12:18:28.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.534+0000 7fc8c559d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc8b804fa80 0x7fc8b804fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.534+0000 7fc8beffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc8b804fa80 0x7fc8b804fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.534+0000 7fc8beffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc8b804fa80 0x7fc8b804fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53376/0 (socket says 172.21.15.52:53376) 2024-05-04T12:18:28.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.534+0000 7fc8beffd640 1 -- 172.21.15.52:0/2914105956 learned_addr learned my addr 172.21.15.52:0/2914105956 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.534+0000 7f69e6f85640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f69d80df390 0x7f69d8007190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.534+0000 7f69e6f85640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f69d804f5c0 0x7f69d8038020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69e6f85640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69d8054390 con 0x7f69d804fa80 2024-05-04T12:18:28.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69e6f85640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f69d8005ea0 con 0x7f69d80df390 2024-05-04T12:18:28.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69e6f85640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f69d8006d00 con 0x7f69d804f5c0 2024-05-04T12:18:28.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69e4d21640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f69d804fa80 0x7f69d804fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.52:59580/0 (socket says 172.21.15.52:59580) 2024-05-04T12:18:28.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69e4d21640 1 -- 172.21.15.52:0/2759066206 learned_addr learned my addr 172.21.15.52:0/2759066206 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:28.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc8c559d640 1 --2- 172.21.15.52:0/2914105956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8b804f5c0 0x7fc8b8038020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69dffff640 1 --2- 172.21.15.52:0/2759066206 >> v2:172.21.15.52:3300/0 conn(0x7f69d80df390 0x7f69d8007190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69e5522640 1 --2- 172.21.15.52:0/2759066206 >> v2:172.21.15.111:3300/0 conn(0x7f69d804f5c0 0x7f69d8038020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc8c559d640 1 -- 172.21.15.52:0/2914105956 --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8b8054390 con 0x7fc8b80df390 2024-05-04T12:18:28.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc8bf7fe640 1 --2- 172.21.15.52:0/2914105956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8b804f5c0 0x7fc8b8038020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69dffff640 1 -- 172.21.15.52:0/2759066206 >> v2:172.21.15.111:3300/0 conn(0x7f69d804f5c0 msgr2=0x7f69d8038020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69dffff640 1 --2- 172.21.15.52:0/2759066206 >> v2:172.21.15.111:3300/0 conn(0x7f69d804f5c0 0x7f69d8038020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc8bdffb640 1 -- 172.21.15.52:0/2914105956 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1855813640 0 0) 0x7fc8b8054390 con 0x7fc8b80df390 2024-05-04T12:18:28.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69dffff640 1 -- 172.21.15.52:0/2759066206 >> v1:172.21.15.111:6789/0 conn(0x7f69d804fa80 legacy=0x7f69d804fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69df7fe640 1 -- 172.21.15.52:0/2759066206 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2992532873 0 0) 0x7f69d8054390 con 0x7f69d804fa80 2024-05-04T12:18:28.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69dffff640 1 -- 172.21.15.52:0/2759066206 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d80060f0 con 0x7f69d80df390 2024-05-04T12:18:28.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc8c559d640 1 -- 172.21.15.52:0/2914105956 --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b8005ea0 con 0x7fc8b804fa80 2024-05-04T12:18:28.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc8c559d640 1 -- 172.21.15.52:0/2914105956 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b8006d00 con 0x7fc8b804f5c0 2024-05-04T12:18:28.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc8bdffb640 1 -- 172.21.15.52:0/2914105956 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8a8004640 con 0x7fc8b80df390 2024-05-04T12:18:28.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc8bf7fe640 1 -- 172.21.15.52:0/2914105956 >> v2:172.21.15.52:3300/0 conn(0x7fc8b804fa80 msgr2=0x7fc8b804fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc8bf7fe640 1 --2- 172.21.15.52:0/2914105956 >> v2:172.21.15.52:3300/0 conn(0x7fc8b804fa80 0x7fc8b804fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc8bf7fe640 1 -- 172.21.15.52:0/2914105956 >> v1:172.21.15.52:6789/0 conn(0x7fc8b80df390 legacy=0x7fc8b8007190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc8bf7fe640 1 -- 172.21.15.52:0/2914105956 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b80503a0 con 0x7fc8b804f5c0 2024-05-04T12:18:28.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fe200bd3640 1 Processor -- start 2024-05-04T12:18:28.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc9f100a640 1 Processor -- start 2024-05-04T12:18:28.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fe200bd3640 1 -- start start 2024-05-04T12:18:28.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f933445c640 1 Processor -- start 2024-05-04T12:18:28.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc9f100a640 1 -- start start 2024-05-04T12:18:28.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc9f100a640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9e4047c20 0x7fc9e40eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc9f100a640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e4034220 con 0x7fc9e4005ea0 2024-05-04T12:18:28.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc9f100a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e404fa30 con 0x7fc9e4033e40 2024-05-04T12:18:28.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc9f100a640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e4050210 con 0x7fc9e4047c20 2024-05-04T12:18:28.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc9eb59e640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc9e4005ea0 0x7fc9e40062f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51500/0 (socket says 172.21.15.52:51500) 2024-05-04T12:18:28.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc9eb59e640 1 -- 172.21.15.52:0/1459555868 learned_addr learned my addr 172.21.15.52:0/1459555868 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:28.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69dffff640 1 --2- 172.21.15.52:0/2759066206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d80df390 0x7f69d8007190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f69d8053140 tx=0x7f69cc01e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5b46985d86458c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fd98465e640 1 Processor -- start 2024-05-04T12:18:28.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fd98465e640 1 -- start start 2024-05-04T12:18:28.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fd98465e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd97c070690 0x7fd97c070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fd98465e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd97c06f4e0 0x7fd97c06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fd98465e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd97c0725f0 con 0x7fd97c14bbf0 2024-05-04T12:18:28.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fd98465e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fd97c06fe20 con 0x7fd97c070690 2024-05-04T12:18:28.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fd98465e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fd97c06e7d0 con 0x7fd97c06f4e0 2024-05-04T12:18:28.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fe200bd3640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe1fc072280 0x7fe1fc0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fe200bd3640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1fc0ccf40 0x7fe1fc15e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fe200bd3640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1fc0cd390 con 0x7fe1fc15c170 2024-05-04T12:18:28.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fe200bd3640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fe1fc0cc7f0 con 0x7fe1fc072280 2024-05-04T12:18:28.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fe200bd3640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe1fc071140 con 0x7fe1fc0ccf40 2024-05-04T12:18:28.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fc9ea59c640 1 --2- 172.21.15.52:0/1459555868 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9e4047c20 0x7fc9e40eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69df7fe640 1 -- 172.21.15.52:0/2759066206 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0ab670 con 0x7f69d80df390 2024-05-04T12:18:28.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69df7fe640 1 -- 172.21.15.52:0/2759066206 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69cc0ab810 con 0x7f69d80df390 2024-05-04T12:18:28.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f69df7fe640 1 -- 172.21.15.52:0/2759066206 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0abbe0 con 0x7f69d80df390 2024-05-04T12:18:28.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fd981bf9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd97c070690 0x7fd97c070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fd982bfb640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd97c06f4e0 0x7fd97c06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fd981bf9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd97c070690 0x7fd97c070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:56966/0 (socket says 172.21.15.52:56966) 2024-05-04T12:18:28.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fd981bf9640 1 -- 172.21.15.52:0/1045632956 learned_addr learned my addr 172.21.15.52:0/1045632956 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fe1f9d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe1fc072280 0x7fe1fc0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fe1f9d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe1fc072280 0x7fe1fc0cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53396/0 (socket says 172.21.15.52:53396) 2024-05-04T12:18:28.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7fe1f9d9b640 1 -- 172.21.15.52:0/915595057 learned_addr learned my addr 172.21.15.52:0/915595057 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fe1fad9d640 1 --2- 172.21.15.52:0/915595057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1fc0ccf40 0x7fe1fc15e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.538+0000 7f933445c640 1 -- start start 2024-05-04T12:18:28.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fd981bf9640 1 -- 172.21.15.52:0/1045632956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd97c06f4e0 msgr2=0x7fd97c06f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f69e6f85640 1 -- 172.21.15.52:0/2759066206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d80df390 msgr2=0x7f69d8007190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f69e6f85640 1 --2- 172.21.15.52:0/2759066206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d80df390 0x7f69d8007190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f69d8053140 tx=0x7f69cc01e680 comp rx=0 tx=0).stop 2024-05-04T12:18:28.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f69e6f85640 1 -- 172.21.15.52:0/2759066206 shutdown_connections 2024-05-04T12:18:28.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f69e6f85640 1 --2- 172.21.15.52:0/2759066206 >> v2:172.21.15.111:3300/0 conn(0x7f69d804f5c0 0x7f69d8038020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f69e6f85640 1 --2- 172.21.15.52:0/2759066206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d80df390 0x7f69d8007190 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f69e6f85640 1 -- 172.21.15.52:0/2759066206 >> 172.21.15.52:0/2759066206 conn(0x7f69d8009fc0 msgr2=0x7f69d800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fc9ea59c640 1 -- 172.21.15.52:0/1459555868 >> v1:172.21.15.111:6789/0 conn(0x7fc9e4033e40 legacy=0x7fc9e40eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f933445c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f932c0cc5e0 0x7f932c070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7ff09b59e640 1 Processor -- start 2024-05-04T12:18:28.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f933445c640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932c0cec10 con 0x7f932c0d03c0 2024-05-04T12:18:28.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f933445c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932c0ceef0 con 0x7f932c0ce730 2024-05-04T12:18:28.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f933445c640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f932c0cf1d0 con 0x7f932c0cc5e0 2024-05-04T12:18:28.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7ff09b59e640 1 -- start start 2024-05-04T12:18:28.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f69e6f85640 1 -- 172.21.15.52:0/2759066206 shutdown_connections 2024-05-04T12:18:28.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fe1f9d9b640 1 -- 172.21.15.52:0/915595057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1fc0ccf40 msgr2=0x7fe1fc15e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fe1f9d9b640 1 --2- 172.21.15.52:0/915595057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1fc0ccf40 0x7fe1fc15e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fe1f9d9b640 1 -- 172.21.15.52:0/915595057 >> v1:172.21.15.52:6789/0 conn(0x7fe1fc15c170 legacy=0x7fe1fc071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fe1f9d9b640 1 -- 172.21.15.52:0/915595057 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1fc070810 con 0x7fe1fc072280 2024-05-04T12:18:28.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f93329f9640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f932c0d03c0 0x7f932c162010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51520/0 (socket says 172.21.15.52:51520) 2024-05-04T12:18:28.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f93329f9640 1 -- 172.21.15.52:0/1922263817 learned_addr learned my addr 172.21.15.52:0/1922263817 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:28.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f69e6f85640 1 -- 172.21.15.52:0/2759066206 wait complete. 2024-05-04T12:18:28.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fd981bf9640 1 --2- 172.21.15.52:0/1045632956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd97c06f4e0 0x7fd97c06f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fd981bf9640 1 -- 172.21.15.52:0/1045632956 >> v1:172.21.15.111:6789/0 conn(0x7fd97c14bbf0 legacy=0x7fd97c1634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fd981bf9640 1 -- 172.21.15.52:0/1045632956 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c06d020 con 0x7fd97c070690 2024-05-04T12:18:28.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fd982bfb640 1 --2- 172.21.15.52:0/1045632956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd97c06f4e0 0x7fd97c06f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:18:28.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f93321f8640 1 --2- 172.21.15.52:0/1922263817 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f932c0cc5e0 0x7f932c070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f93311f6640 1 -- 172.21.15.52:0/1922263817 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3659716198 0 0) 0x7f932c0cec10 con 0x7f932c0d03c0 2024-05-04T12:18:28.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f93311f6640 1 -- 172.21.15.52:0/1922263817 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9318003530 con 0x7f932c0d03c0 2024-05-04T12:18:28.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f93321f8640 1 -- 172.21.15.52:0/1922263817 >> v1:172.21.15.111:6789/0 conn(0x7f932c0ce730 legacy=0x7f932c0cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f93311f6640 1 -- 172.21.15.52:0/1922263817 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 133103850 0 0) 0x7f932c0ceef0 con 0x7f932c0ce730 2024-05-04T12:18:28.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f93321f8640 1 -- 172.21.15.52:0/1922263817 >> v1:172.21.15.52:6789/0 conn(0x7f932c0d03c0 legacy=0x7f932c162010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f93321f8640 1 -- 172.21.15.52:0/1922263817 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c0cf4b0 con 0x7f932c0cc5e0 2024-05-04T12:18:28.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fc9e9d9b640 1 -- 172.21.15.52:0/1459555868 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1652265419 0 0) 0x7fc9e4034220 con 0x7fc9e4005ea0 2024-05-04T12:18:28.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fc9ea59c640 1 -- 172.21.15.52:0/1459555868 >> v1:172.21.15.52:6789/0 conn(0x7fc9e4005ea0 legacy=0x7fc9e40062f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fc9ea59c640 1 -- 172.21.15.52:0/1459555868 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e404f2e0 con 0x7fc9e4047c20 2024-05-04T12:18:28.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f69e6f85640 1 Processor -- start 2024-05-04T12:18:28.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7ff09b59e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff09c145b10 0x7ff09c15c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f623f7c6640 1 Processor -- start 2024-05-04T12:18:28.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fd981bf9640 1 --2- 172.21.15.52:0/1045632956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd97c070690 0x7fd97c070b20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009870 tx=0x7fd96c01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdc89b8323946070 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fe1f9d9b640 1 --2- 172.21.15.52:0/915595057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1fc072280 0x7fe1fc0cca00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009870 tx=0x7fe1ec01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc40c95ad1e2e9c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f623f7c6640 1 -- start start 2024-05-04T12:18:28.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7f69e6f85640 1 -- start start 2024-05-04T12:18:28.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.542+0000 7fc8bf7fe640 1 --2- 172.21.15.52:0/2914105956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8b804f5c0 0x7fc8b8038020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0050e50 tx=0x7fc8c0097220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6705ea88cebb56e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fd9813f8640 1 -- 172.21.15.52:0/1045632956 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0ab650 con 0x7fd97c070690 2024-05-04T12:18:28.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff099d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff09c145b10 0x7ff09c15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff099d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff09c145b10 0x7ff09c15c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53404/0 (socket says 172.21.15.52:53404) 2024-05-04T12:18:28.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fd9813f8640 1 -- 172.21.15.52:0/1045632956 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd96c0b0070 con 0x7fd97c070690 2024-05-04T12:18:28.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fd9813f8640 1 -- 172.21.15.52:0/1045632956 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0aba10 con 0x7fd97c070690 2024-05-04T12:18:28.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff09a59c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff09c145730 0x7ff09c071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.52:59612/0 (socket says 172.21.15.52:59612) 2024-05-04T12:18:28.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fe1f959a640 1 -- 172.21.15.52:0/915595057 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0b5070 con 0x7fe1fc072280 2024-05-04T12:18:28.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fe1f959a640 1 -- 172.21.15.52:0/915595057 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ec0a8a50 con 0x7fe1fc072280 2024-05-04T12:18:28.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fe1f959a640 1 -- 172.21.15.52:0/915595057 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0a8e20 con 0x7fe1fc072280 2024-05-04T12:18:28.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8bdffb640 1 -- 172.21.15.52:0/2914105956 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c012d070 con 0x7fc8b804f5c0 2024-05-04T12:18:28.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff099d9b640 1 -- 172.21.15.52:0/2618451482 learned_addr learned my addr 172.21.15.52:0/2618451482 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff09b59e640 1 -- 172.21.15.52:0/2618451482 --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff09c0708a0 con 0x7ff09c072280 2024-05-04T12:18:28.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69e6f85640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f69d804f5c0 0x7f69d80dda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69e6f85640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d804fa80 0x7f69d80d7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69e6f85640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69d80df390 0x7f69d80d81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69e6f85640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f69d80076d0 con 0x7f69d804fa80 2024-05-04T12:18:28.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69e6f85640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f69d80ebfd0 con 0x7f69d80df390 2024-05-04T12:18:28.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69e6f85640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f69d80daad0 con 0x7f69d804f5c0 2024-05-04T12:18:28.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff09b59e640 1 -- 172.21.15.52:0/2618451482 --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff09c070b80 con 0x7ff09c145730 2024-05-04T12:18:28.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff09b59e640 1 -- 172.21.15.52:0/2618451482 --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ff09c070e60 con 0x7ff09c145b10 2024-05-04T12:18:28.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69e4d21640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f69d804f5c0 0x7f69d80dda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69dffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d804fa80 0x7f69d80d7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fd98465e640 1 -- 172.21.15.52:0/1045632956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd97c070690 msgr2=0x7fd97c070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fd98465e640 1 --2- 172.21.15.52:0/1045632956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd97c070690 0x7fd97c070b20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009870 tx=0x7fd96c01c560 comp rx=0 tx=0).stop 2024-05-04T12:18:28.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fe200bd3640 1 -- 172.21.15.52:0/915595057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1fc072280 msgr2=0x7fe1fc0cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fe200bd3640 1 --2- 172.21.15.52:0/915595057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1fc072280 0x7fe1fc0cca00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009870 tx=0x7fe1ec01ed50 comp rx=0 tx=0).stop 2024-05-04T12:18:28.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fe200bd3640 1 -- 172.21.15.52:0/915595057 shutdown_connections 2024-05-04T12:18:28.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fe200bd3640 1 --2- 172.21.15.52:0/915595057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1fc0ccf40 0x7fe1fc15e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fe200bd3640 1 --2- 172.21.15.52:0/915595057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1fc072280 0x7fe1fc0cca00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fe200bd3640 1 -- 172.21.15.52:0/915595057 >> 172.21.15.52:0/915595057 conn(0x7fe1fc073180 msgr2=0x7fe1fc0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69dffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d804fa80 0x7f69d80d7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53412/0 (socket says 172.21.15.52:53412) 2024-05-04T12:18:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69dffff640 1 -- 172.21.15.52:0/3323135127 learned_addr learned my addr 172.21.15.52:0/3323135127 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69e5522640 1 --2- 172.21.15.52:0/3323135127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69d80df390 0x7f69d80d81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f93321f8640 1 --2- 172.21.15.52:0/1922263817 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f932c0cc5e0 0x7f932c070fe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f93280097f0 tx=0x7f932801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1c6c0a3d80b0313 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff09959a640 1 -- 172.21.15.52:0/2618451482 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2580666755 0 0) 0x7ff09c070b80 con 0x7ff09c145730 2024-05-04T12:18:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff09959a640 1 -- 172.21.15.52:0/2618451482 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff084003530 con 0x7ff09c145730 2024-05-04T12:18:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff09959a640 1 -- 172.21.15.52:0/2618451482 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 320034812 0 0) 0x7ff09c0708a0 con 0x7ff09c072280 2024-05-04T12:18:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff09959a640 1 -- 172.21.15.52:0/2618451482 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff09c070b80 con 0x7ff09c072280 2024-05-04T12:18:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8bdffb640 1 -- 172.21.15.52:0/2914105956 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c0120a20 con 0x7fc8b804f5c0 2024-05-04T12:18:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8bdffb640 1 -- 172.21.15.52:0/2914105956 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0128440 con 0x7fc8b804f5c0 2024-05-04T12:18:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 -- 172.21.15.52:0/2914105956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8b804f5c0 msgr2=0x7fc8b8038020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 --2- 172.21.15.52:0/2914105956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8b804f5c0 0x7fc8b8038020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0050e50 tx=0x7fc8c0097220 comp rx=0 tx=0).stop 2024-05-04T12:18:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fd98465e640 1 -- 172.21.15.52:0/1045632956 shutdown_connections 2024-05-04T12:18:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fd98465e640 1 --2- 172.21.15.52:0/1045632956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd97c06f4e0 0x7fd97c06f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fd98465e640 1 --2- 172.21.15.52:0/1045632956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd97c070690 0x7fd97c070b20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fd98465e640 1 -- 172.21.15.52:0/1045632956 >> 172.21.15.52:0/1045632956 conn(0x7fd97c072d80 msgr2=0x7fd97c073190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 -- 172.21.15.52:0/2914105956 shutdown_connections 2024-05-04T12:18:28.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 --2- 172.21.15.52:0/2914105956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8b804f5c0 0x7fc8b8038020 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 --2- 172.21.15.52:0/2914105956 >> v2:172.21.15.52:3300/0 conn(0x7fc8b804fa80 0x7fc8b804fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 -- 172.21.15.52:0/2914105956 >> 172.21.15.52:0/2914105956 conn(0x7fc8b8009fc0 msgr2=0x7fc8b800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fd98465e640 1 -- 172.21.15.52:0/1045632956 shutdown_connections 2024-05-04T12:18:28.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 -- 172.21.15.52:0/2914105956 shutdown_connections 2024-05-04T12:18:28.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff099d9b640 1 -- 172.21.15.52:0/2618451482 >> v1:172.21.15.111:6789/0 conn(0x7ff09c145730 legacy=0x7ff09c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff099d9b640 1 -- 172.21.15.52:0/2618451482 >> v1:172.21.15.52:6789/0 conn(0x7ff09c072280 legacy=0x7ff09c0ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fe200bd3640 1 -- 172.21.15.52:0/915595057 shutdown_connections 2024-05-04T12:18:28.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff09959a640 1 -- 172.21.15.52:0/2618451482 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2596520014 0 0) 0x7ff09c070b80 con 0x7ff09c072280 2024-05-04T12:18:28.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7ff099d9b640 1 -- 172.21.15.52:0/2618451482 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c071140 con 0x7ff09c145b10 2024-05-04T12:18:28.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f93311f6640 1 -- 172.21.15.52:0/1922263817 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c0af9e0 con 0x7f932c0cc5e0 2024-05-04T12:18:28.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f93311f6640 1 -- 172.21.15.52:0/1922263817 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93280af070 con 0x7f932c0cc5e0 2024-05-04T12:18:28.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f93311f6640 1 -- 172.21.15.52:0/1922263817 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280aa630 con 0x7f932c0cc5e0 2024-05-04T12:18:28.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc9ea59c640 1 --2- 172.21.15.52:0/1459555868 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9e4047c20 0x7fc9e40eb890 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00097f0 tx=0x7fc9e001ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0f6a6885e86bfab server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 -- 172.21.15.52:0/2914105956 wait complete. 2024-05-04T12:18:28.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f933445c640 1 -- 172.21.15.52:0/1922263817 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f932c0cc5e0 msgr2=0x7f932c070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f933445c640 1 --2- 172.21.15.52:0/1922263817 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f932c0cc5e0 0x7f932c070fe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f93280097f0 tx=0x7f932801c4b0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 Processor -- start 2024-05-04T12:18:28.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69dffff640 1 -- 172.21.15.52:0/3323135127 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f69d804f5c0 msgr2=0x7f69d80dda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69dffff640 1 --2- 172.21.15.52:0/3323135127 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f69d804f5c0 0x7f69d80dda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69dffff640 1 -- 172.21.15.52:0/3323135127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69d80df390 msgr2=0x7f69d80d81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69dffff640 1 --2- 172.21.15.52:0/3323135127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69d80df390 0x7f69d80d81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69dffff640 1 -- 172.21.15.52:0/3323135127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d80d3db0 con 0x7f69d804fa80 2024-05-04T12:18:28.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69dffff640 1 --2- 172.21.15.52:0/3323135127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d804fa80 0x7f69d80d7cb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f69cc01b290 tx=0x7f69cc01cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f933445c640 1 -- 172.21.15.52:0/1922263817 shutdown_connections 2024-05-04T12:18:28.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f933445c640 1 --2- 172.21.15.52:0/1922263817 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f932c0cc5e0 0x7f932c070fe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f933445c640 1 -- 172.21.15.52:0/1922263817 >> 172.21.15.52:0/1922263817 conn(0x7f932c0b2210 msgr2=0x7f932c0b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fe200bd3640 1 -- 172.21.15.52:0/915595057 wait complete. 2024-05-04T12:18:28.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f933445c640 1 -- 172.21.15.52:0/1922263817 shutdown_connections 2024-05-04T12:18:28.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fe200bd3640 1 Processor -- start 2024-05-04T12:18:28.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f933445c640 1 -- 172.21.15.52:0/1922263817 wait complete. 2024-05-04T12:18:28.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69dd7fa640 1 -- 172.21.15.52:0/3323135127 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0be680 con 0x7f69d804fa80 2024-05-04T12:18:28.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69dd7fa640 1 -- 172.21.15.52:0/3323135127 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69cc0be820 con 0x7f69d804fa80 2024-05-04T12:18:28.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 -- start start 2024-05-04T12:18:28.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc9e9d9b640 1 -- 172.21.15.52:0/1459555868 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00b5070 con 0x7fc9e4047c20 2024-05-04T12:18:28.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7f69dd7fa640 1 -- 172.21.15.52:0/3323135127 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0bebd0 con 0x7f69d804fa80 2024-05-04T12:18:28.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fe200bd3640 1 -- start start 2024-05-04T12:18:28.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc9e9d9b640 1 -- 172.21.15.52:0/1459555868 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e00a89d0 con 0x7fc9e4047c20 2024-05-04T12:18:28.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc9e9d9b640 1 -- 172.21.15.52:0/1459555868 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00a8da0 con 0x7fc9e4047c20 2024-05-04T12:18:28.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fd98465e640 1 -- 172.21.15.52:0/1045632956 wait complete. 2024-05-04T12:18:28.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fd98465e640 1 Processor -- start 2024-05-04T12:18:28.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8b804f5c0 0x7fc8b804a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8b804fa80 0x7fc8b81844f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8b80df390 0x7fc8b80cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b80ebfd0 con 0x7fc8b80df390 2024-05-04T12:18:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b8006990 con 0x7fc8b804fa80 2024-05-04T12:18:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.546+0000 7fc8c559d640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b80ec150 con 0x7fc8b804f5c0 2024-05-04T12:18:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc8bf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8b80df390 0x7fc8b80cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc8bf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8b80df390 0x7fc8b80cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53414/0 (socket says 172.21.15.52:53414) 2024-05-04T12:18:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc8bf7fe640 1 -- 172.21.15.52:0/462505299 learned_addr learned my addr 172.21.15.52:0/462505299 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 -- 172.21.15.52:0/1459555868 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9e4047c20 msgr2=0x7fc9e40eb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 --2- 172.21.15.52:0/1459555868 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9e4047c20 0x7fc9e40eb890 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00097f0 tx=0x7fc9e001ecd0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc8bf7fe640 1 -- 172.21.15.52:0/462505299 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8b804f5c0 msgr2=0x7fc8b804a660 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-04T12:18:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc8bf7fe640 1 --2- 172.21.15.52:0/462505299 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8b804f5c0 0x7fc8b804a660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc8bf7fe640 1 -- 172.21.15.52:0/462505299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8b804fa80 msgr2=0x7fc8b81844f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-04T12:18:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc8bf7fe640 1 --2- 172.21.15.52:0/462505299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8b804fa80 0x7fc8b81844f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc8bf7fe640 1 -- 172.21.15.52:0/462505299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b8184ed0 con 0x7fc8b80df390 2024-05-04T12:18:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 -- 172.21.15.52:0/1459555868 shutdown_connections 2024-05-04T12:18:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 --2- 172.21.15.52:0/1459555868 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9e4047c20 0x7fc9e40eb890 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 -- 172.21.15.52:0/1459555868 >> 172.21.15.52:0/1459555868 conn(0x7fc9e4009fc0 msgr2=0x7fc9e400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 -- 172.21.15.52:0/1459555868 shutdown_connections 2024-05-04T12:18:28.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc8bf7fe640 1 --2- 172.21.15.52:0/462505299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8b80df390 0x7fc8b80cf260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc8c006a8f0 tx=0x7fc8c0093fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f623f7c6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f62380b28e0 0x7f62380b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f623f7c6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f623815a210 0x7f623806df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f69e6f85640 1 -- 172.21.15.52:0/3323135127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d80d4090 con 0x7f69d804fa80 2024-05-04T12:18:28.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f69e6f85640 1 -- 172.21.15.52:0/3323135127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d80ddfa0 con 0x7f69d804fa80 2024-05-04T12:18:28.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f623f7c6640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f623806e8d0 con 0x7f623806e4c0 2024-05-04T12:18:28.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f623f7c6640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f62380b12a0 con 0x7f623815a210 2024-05-04T12:18:28.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f623f7c6640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f623815e250 con 0x7f62380b28e0 2024-05-04T12:18:28.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f933445c640 1 Processor -- start 2024-05-04T12:18:28.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f623d562640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f62380b28e0 0x7f62380b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f623d562640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f62380b28e0 0x7f62380b0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:56998/0 (socket says 172.21.15.52:56998) 2024-05-04T12:18:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f623d562640 1 -- 172.21.15.52:0/2741192402 learned_addr learned my addr 172.21.15.52:0/2741192402 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f933445c640 1 -- start start 2024-05-04T12:18:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f623d562640 1 -- 172.21.15.52:0/2741192402 >> v2:172.21.15.52:3300/0 conn(0x7f623815a210 msgr2=0x7f623806df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f623d562640 1 --2- 172.21.15.52:0/2741192402 >> v2:172.21.15.52:3300/0 conn(0x7f623815a210 0x7f623806df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f623d562640 1 -- 172.21.15.52:0/2741192402 >> v1:172.21.15.52:6789/0 conn(0x7f623806e4c0 legacy=0x7f623806cd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T12:18:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f623d562640 1 -- 172.21.15.52:0/2741192402 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623815e530 con 0x7f62380b28e0 2024-05-04T12:18:28.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe200bd3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1fc072280 0x7fe1fc1476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe200bd3640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1fc0ccf40 0x7fe1fc148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe200bd3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1fc15c170 0x7fe1fc145bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe200bd3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc15b3a0 con 0x7fe1fc15c170 2024-05-04T12:18:28.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe200bd3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc070dd0 con 0x7fe1fc072280 2024-05-04T12:18:28.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe200bd3640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe1fc15b520 con 0x7fe1fc0ccf40 2024-05-04T12:18:28.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7ff099d9b640 1 --2- 172.21.15.52:0/2618451482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c145b10 0x7ff09c15c360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff090009870 tx=0x7ff09001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9443ba5083ed7065 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe1fa59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1fc072280 0x7fe1fc1476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe1fa59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1fc072280 0x7fe1fc1476e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:57000/0 (socket says 172.21.15.52:57000) 2024-05-04T12:18:28.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe1fa59c640 1 -- 172.21.15.52:0/1001136053 learned_addr learned my addr 172.21.15.52:0/1001136053 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f69e6f85640 1 -- 172.21.15.52:0/3323135127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69d80db100 con 0x7f69d804fa80 2024-05-04T12:18:28.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 -- 172.21.15.52:0/1459555868 wait complete. 2024-05-04T12:18:28.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe1f9d9b640 1 --2- 172.21.15.52:0/1001136053 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1fc0ccf40 0x7fe1fc148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 Processor -- start 2024-05-04T12:18:28.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f933445c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f932c0cc5e0 0x7f932c149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe1fa59c640 1 -- 172.21.15.52:0/1001136053 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1fc0ccf40 msgr2=0x7fe1fc148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe1fa59c640 1 --2- 172.21.15.52:0/1001136053 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1fc0ccf40 0x7fe1fc148f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe1fa59c640 1 -- 172.21.15.52:0/1001136053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1fc15c170 msgr2=0x7fe1fc145bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe1fa59c640 1 --2- 172.21.15.52:0/1001136053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1fc15c170 0x7fe1fc145bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f933445c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f932c0ce730 0x7f932c149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe1fa59c640 1 -- 172.21.15.52:0/1001136053 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1fc1494c0 con 0x7fe1fc072280 2024-05-04T12:18:28.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f933445c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932c0d03c0 0x7f932c06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f93321f8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f932c0cc5e0 0x7f932c149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 -- start start 2024-05-04T12:18:28.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f933445c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f932c071520 con 0x7f932c0d03c0 2024-05-04T12:18:28.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f93321f8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f932c0cc5e0 0x7f932c149650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:57012/0 (socket says 172.21.15.52:57012) 2024-05-04T12:18:28.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f93321f8640 1 -- 172.21.15.52:0/2499366432 learned_addr learned my addr 172.21.15.52:0/2499366432 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f933445c640 1 -- 172.21.15.52:0/2499366432 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f932c0ccc10 con 0x7f932c0cc5e0 2024-05-04T12:18:28.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f933445c640 1 -- 172.21.15.52:0/2499366432 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f932c0716a0 con 0x7f932c0ce730 2024-05-04T12:18:28.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7ff09959a640 1 -- 172.21.15.52:0/2618451482 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0900b0070 con 0x7ff09c145b10 2024-05-04T12:18:28.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f93319f7640 1 --2- 172.21.15.52:0/2499366432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f932c0ce730 0x7f932c149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f93329f9640 1 --2- 172.21.15.52:0/2499366432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932c0d03c0 0x7f932c06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fd98465e640 1 -- start start 2024-05-04T12:18:28.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f93329f9640 1 -- 172.21.15.52:0/2499366432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f932c0ce730 msgr2=0x7f932c149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f93329f9640 1 --2- 172.21.15.52:0/2499366432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f932c0ce730 0x7f932c149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f93329f9640 1 -- 172.21.15.52:0/2499366432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f932c0cc5e0 msgr2=0x7f932c149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f93329f9640 1 --2- 172.21.15.52:0/2499366432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f932c0cc5e0 0x7f932c149650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f93329f9640 1 -- 172.21.15.52:0/2499366432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c14fb70 con 0x7f932c0d03c0 2024-05-04T12:18:28.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f93329f9640 1 --2- 172.21.15.52:0/2499366432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932c0d03c0 0x7f932c06d8d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f932c0676d0 tx=0x7f932400ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe1fa59c640 1 --2- 172.21.15.52:0/1001136053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1fc072280 0x7fe1fc1476e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000c3c0 tx=0x7fe1f000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe1eaffd640 1 -- 172.21.15.52:0/1001136053 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0017070 con 0x7fe1fc072280 2024-05-04T12:18:28.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe200bd3640 1 -- 172.21.15.52:0/1001136053 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1fc14aaf0 con 0x7fe1fc072280 2024-05-04T12:18:28.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7ff09959a640 1 -- 172.21.15.52:0/2618451482 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0900ab400 con 0x7ff09c145b10 2024-05-04T12:18:28.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7ff09959a640 1 -- 172.21.15.52:0/2618451482 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094001a00 con 0x7ff09c145b10 2024-05-04T12:18:28.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fe200bd3640 1 -- 172.21.15.52:0/1001136053 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1fc14b0b0 con 0x7fe1fc072280 2024-05-04T12:18:28.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fd98465e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd97c06f4e0 0x7fd97c15bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7ff09b59e640 1 -- 172.21.15.52:0/2618451482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c145b10 msgr2=0x7ff09c15c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7ff09b59e640 1 --2- 172.21.15.52:0/2618451482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c145b10 0x7ff09c15c360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff090009870 tx=0x7ff09001c3f0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7ff09b59e640 1 -- 172.21.15.52:0/2618451482 shutdown_connections 2024-05-04T12:18:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7ff09b59e640 1 --2- 172.21.15.52:0/2618451482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c145b10 0x7ff09c15c360 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7ff09b59e640 1 -- 172.21.15.52:0/2618451482 >> 172.21.15.52:0/2618451482 conn(0x7ff09c067440 msgr2=0x7ff09c0b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9e4005ea0 0x7fc9e404a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9e4033e40 0x7fc9e40dc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9e4047c20 0x7fc9e40485d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e404f090 con 0x7fc9e4033e40 2024-05-04T12:18:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e404fd80 con 0x7fc9e4047c20 2024-05-04T12:18:28.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7fc9f100a640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e40482c0 con 0x7fc9e4005ea0 2024-05-04T12:18:28.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fe1e8ff9640 1 -- 172.21.15.52:0/1001136053 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe1e0001040 con 0x7fe1fc072280 2024-05-04T12:18:28.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc8a77fe640 1 -- 172.21.15.52:0/462505299 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c012d040 con 0x7fc8b80df390 2024-05-04T12:18:28.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc8a77fe640 1 -- 172.21.15.52:0/462505299 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c0133640 con 0x7fc8b80df390 2024-05-04T12:18:28.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc8a77fe640 1 -- 172.21.15.52:0/462505299 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c01339f0 con 0x7fc8b80df390 2024-05-04T12:18:28.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc8c559d640 1 -- 172.21.15.52:0/462505299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b81851b0 con 0x7fc8b80df390 2024-05-04T12:18:28.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc8c559d640 1 -- 172.21.15.52:0/462505299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b804aba0 con 0x7fc8b80df390 2024-05-04T12:18:28.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fd9823fa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd97c06f4e0 0x7fd97c15bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fd9823fa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd97c06f4e0 0x7fd97c15bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:57026/0 (socket says 172.21.15.52:57026) 2024-05-04T12:18:28.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7ff09b59e640 1 -- 172.21.15.52:0/2618451482 shutdown_connections 2024-05-04T12:18:28.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.550+0000 7f69dd7fa640 1 -- 172.21.15.52:0/3323135127 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f69cc0c73f0 con 0x7f69d804fa80 2024-05-04T12:18:28.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f69dd7fa640 1 --2- 172.21.15.52:0/3323135127 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f69c4067990 0x7f69c4069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f69dd7fa640 1 -- 172.21.15.52:0/3323135127 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f69cc121450 con 0x7f69d804fa80 2024-05-04T12:18:28.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f69dd7fa640 1 --2- 172.21.15.52:0/3323135127 >> [v2:172.21.15.111:6823/2919151955,v1:172.21.15.111:6825/2919151955] conn(0x7f69c406fbb0 0x7f69c4071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f69dd7fa640 1 -- 172.21.15.52:0/3323135127 --> [v2:172.21.15.111:6823/2919151955,v1:172.21.15.111:6825/2919151955] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69cc001d40 con 0x7f69c406fbb0 2024-05-04T12:18:28.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f69dd7fa640 1 -- 172.21.15.52:0/3323135127 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f69cc121840 con 0x7f69d804fa80 2024-05-04T12:18:28.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f623d562640 1 --2- 172.21.15.52:0/2741192402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62380b28e0 0x7f62380b0d60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f62380cdd20 tx=0x7f623401c320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca4348eb65c6f689 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f69e4d21640 1 --2- 172.21.15.52:0/3323135127 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f69c4067990 0x7f69c4069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9eb59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9e4047c20 0x7fc9e40485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9eb59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9e4047c20 0x7fc9e40485d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:57014/0 (socket says 172.21.15.52:57014) 2024-05-04T12:18:28.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9eb59e640 1 -- 172.21.15.52:0/3968926859 learned_addr learned my addr 172.21.15.52:0/3968926859 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9ea59c640 1 --2- 172.21.15.52:0/3968926859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9e4033e40 0x7fc9e40dc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9ead9d640 1 --2- 172.21.15.52:0/3968926859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9e4005ea0 0x7fc9e404a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f69e5522640 1 --2- 172.21.15.52:0/3323135127 >> [v2:172.21.15.111:6823/2919151955,v1:172.21.15.111:6825/2919151955] conn(0x7f69c406fbb0 0x7f69c4071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f622ffff640 1 -- 172.21.15.52:0/2741192402 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62340b0070 con 0x7f62380b28e0 2024-05-04T12:18:28.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f622ffff640 1 -- 172.21.15.52:0/2741192402 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62340ab600 con 0x7f62380b28e0 2024-05-04T12:18:28.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f622ffff640 1 -- 172.21.15.52:0/2741192402 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62340aba10 con 0x7f62380b28e0 2024-05-04T12:18:28.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f9322ffd640 1 -- 172.21.15.52:0/2499366432 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324017070 con 0x7f932c0d03c0 2024-05-04T12:18:28.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f933445c640 1 -- 172.21.15.52:0/2499366432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c14fe50 con 0x7f932c0d03c0 2024-05-04T12:18:28.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f933445c640 1 -- 172.21.15.52:0/2499366432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c154520 con 0x7f932c0d03c0 2024-05-04T12:18:28.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f9322ffd640 1 -- 172.21.15.52:0/2499366432 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f932400ab10 con 0x7f932c0d03c0 2024-05-04T12:18:28.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f9322ffd640 1 -- 172.21.15.52:0/2499366432 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324012400 con 0x7f932c0d03c0 2024-05-04T12:18:28.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f933445c640 1 -- 172.21.15.52:0/2499366432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f932c14a0d0 con 0x7f932c0d03c0 2024-05-04T12:18:28.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9ea59c640 1 -- 172.21.15.52:0/3968926859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9e4005ea0 msgr2=0x7fc9e404a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9ea59c640 1 --2- 172.21.15.52:0/3968926859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9e4005ea0 0x7fc9e404a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9ea59c640 1 -- 172.21.15.52:0/3968926859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9e4047c20 msgr2=0x7fc9e40485d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9ea59c640 1 --2- 172.21.15.52:0/3968926859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9e4047c20 0x7fc9e40485d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9ea59c640 1 -- 172.21.15.52:0/3968926859 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e4048e20 con 0x7fc9e4033e40 2024-05-04T12:18:28.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f69e4d21640 1 --2- 172.21.15.52:0/3323135127 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f69c4067990 0x7f69c4069e50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f69d4000f70 tx=0x7f69d4007240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7ff09b59e640 1 -- 172.21.15.52:0/2618451482 wait complete. 2024-05-04T12:18:28.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9ea59c640 1 --2- 172.21.15.52:0/3968926859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9e4033e40 0x7fc9e40dc2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00af000 tx=0x7fc9e001ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fd98465e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd97c070690 0x7fd97c15d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fd98465e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd97c14bbf0 0x7fd97c0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc8c559d640 1 -- 172.21.15.52:0/462505299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc8b81853f0 con 0x7fc8b80df390 2024-05-04T12:18:28.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7ff09b59e640 1 Processor -- start 2024-05-04T12:18:28.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f623f7c6640 1 -- 172.21.15.52:0/2741192402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62380b28e0 msgr2=0x7f62380b0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f623f7c6640 1 --2- 172.21.15.52:0/2741192402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62380b28e0 0x7f62380b0d60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f62380cdd20 tx=0x7f623401c320 comp rx=0 tx=0).stop 2024-05-04T12:18:28.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fd982bfb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd97c14bbf0 0x7fd97c0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fd981bf9640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd97c070690 0x7fd97c15d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fd9823fa640 1 -- 172.21.15.52:0/80853820 learned_addr learned my addr 172.21.15.52:0/80853820 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fd98465e640 1 -- 172.21.15.52:0/80853820 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c14c150 con 0x7fd97c14bbf0 2024-05-04T12:18:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fd98465e640 1 -- 172.21.15.52:0/80853820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c14bfd0 con 0x7fd97c06f4e0 2024-05-04T12:18:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fd98465e640 1 -- 172.21.15.52:0/80853820 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fd97c06cbd0 con 0x7fd97c070690 2024-05-04T12:18:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fe1eaffd640 1 -- 172.21.15.52:0/1001136053 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f00052d0 con 0x7fe1fc072280 2024-05-04T12:18:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fe1eaffd640 1 -- 172.21.15.52:0/1001136053 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0003840 con 0x7fe1fc072280 2024-05-04T12:18:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f623f7c6640 1 -- 172.21.15.52:0/2741192402 shutdown_connections 2024-05-04T12:18:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f623f7c6640 1 --2- 172.21.15.52:0/2741192402 >> v2:172.21.15.52:3300/0 conn(0x7f623815a210 0x7f623806df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f623f7c6640 1 --2- 172.21.15.52:0/2741192402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62380b28e0 0x7f62380b0d60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f623f7c6640 1 -- 172.21.15.52:0/2741192402 >> 172.21.15.52:0/2741192402 conn(0x7f6238067070 msgr2=0x7f6238067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9d37fe640 1 -- 172.21.15.52:0/3968926859 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00b5040 con 0x7fc9e4033e40 2024-05-04T12:18:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9d37fe640 1 -- 172.21.15.52:0/3968926859 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e00c0070 con 0x7fc9e4033e40 2024-05-04T12:18:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9d37fe640 1 -- 172.21.15.52:0/3968926859 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00b1bc0 con 0x7fc9e4033e40 2024-05-04T12:18:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9f100a640 1 -- 172.21.15.52:0/3968926859 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e4049100 con 0x7fc9e4033e40 2024-05-04T12:18:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7fc9f100a640 1 -- 172.21.15.52:0/3968926859 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e404bd40 con 0x7fc9e4033e40 2024-05-04T12:18:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7ff09b59e640 1 -- start start 2024-05-04T12:18:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f9322ffd640 1 -- 172.21.15.52:0/2499366432 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f932401a070 con 0x7f932c0d03c0 2024-05-04T12:18:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f9322ffd640 1 --2- 172.21.15.52:0/2499366432 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f93100678c0 0x7f9310069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f9322ffd640 1 -- 172.21.15.52:0/2499366432 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f93240858b0 con 0x7f932c0d03c0 2024-05-04T12:18:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f93321f8640 1 --2- 172.21.15.52:0/2499366432 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f93100678c0 0x7f9310069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f9322ffd640 1 --2- 172.21.15.52:0/2499366432 >> [v2:172.21.15.111:6803/1318681540,v1:172.21.15.111:6807/1318681540] conn(0x7f931006fa60 0x7f9310071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f9322ffd640 1 -- 172.21.15.52:0/2499366432 --> [v2:172.21.15.111:6803/1318681540,v1:172.21.15.111:6807/1318681540] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9310072530 con 0x7f931006fa60 2024-05-04T12:18:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.554+0000 7f69e5522640 1 --2- 172.21.15.52:0/3323135127 >> [v2:172.21.15.111:6823/2919151955,v1:172.21.15.111:6825/2919151955] conn(0x7f69c406fbb0 0x7f69c4071fd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fd9823fa640 1 -- 172.21.15.52:0/80853820 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd97c070690 msgr2=0x7fd97c15d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fd9823fa640 1 --2- 172.21.15.52:0/80853820 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd97c070690 0x7fd97c15d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fd9823fa640 1 -- 172.21.15.52:0/80853820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd97c14bbf0 msgr2=0x7fd97c0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fd9823fa640 1 --2- 172.21.15.52:0/80853820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd97c14bbf0 0x7fd97c0cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fd9823fa640 1 -- 172.21.15.52:0/80853820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c15d9b0 con 0x7fd97c06f4e0 2024-05-04T12:18:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7f93321f8640 1 --2- 172.21.15.52:0/2499366432 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f93100678c0 0x7f9310069d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9328000c10 tx=0x7f93280b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7f9322ffd640 1 -- 172.21.15.52:0/2499366432 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9324051fa0 con 0x7f932c0d03c0 2024-05-04T12:18:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fe1eaffd640 1 -- 172.21.15.52:0/1001136053 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe1f001a070 con 0x7fe1fc072280 2024-05-04T12:18:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff09b59e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff09c145730 0x7ff09c15ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff09b59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c145b10 0x7ff09c132d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff09b59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff09c139770 0x7ff09c139bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff09b59e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c1603f0 con 0x7ff09c145b10 2024-05-04T12:18:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff09b59e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c160570 con 0x7ff09c139770 2024-05-04T12:18:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff09b59e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7ff09c1606f0 con 0x7ff09c145730 2024-05-04T12:18:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff099d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c145b10 0x7ff09c132d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff099d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c145b10 0x7ff09c132d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53470/0 (socket says 172.21.15.52:53470) 2024-05-04T12:18:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff099d9b640 1 -- 172.21.15.52:0/279597511 learned_addr learned my addr 172.21.15.52:0/279597511 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc8a77fe640 1 -- 172.21.15.52:0/462505299 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc8c013a060 con 0x7fc8b80df390 2024-05-04T12:18:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc8a77fe640 1 --2- 172.21.15.52:0/462505299 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc8ac0679c0 0x7fc8ac069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc8a77fe640 1 -- 172.21.15.52:0/462505299 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc8c019b290 con 0x7fc8b80df390 2024-05-04T12:18:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc8a77fe640 1 --2- 172.21.15.52:0/462505299 >> [v2:172.21.15.52:6808/2539299307,v1:172.21.15.52:6809/2539299307] conn(0x7fc8ac06fbe0 0x7fc8ac072000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc8a77fe640 1 -- 172.21.15.52:0/462505299 --> [v2:172.21.15.52:6808/2539299307,v1:172.21.15.52:6809/2539299307] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc8c006e0f0 con 0x7fc8ac06fbe0 2024-05-04T12:18:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc8a77fe640 1 -- 172.21.15.52:0/462505299 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc8c019b680 con 0x7fc8b80df390 2024-05-04T12:18:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc8be7fc640 1 --2- 172.21.15.52:0/462505299 >> [v2:172.21.15.52:6808/2539299307,v1:172.21.15.52:6809/2539299307] conn(0x7fc8ac06fbe0 0x7fc8ac072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc8beffd640 1 --2- 172.21.15.52:0/462505299 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc8ac0679c0 0x7fc8ac069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc8be7fc640 1 --2- 172.21.15.52:0/462505299 >> [v2:172.21.15.52:6808/2539299307,v1:172.21.15.52:6809/2539299307] conn(0x7fc8ac06fbe0 0x7fc8ac072000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc8beffd640 1 --2- 172.21.15.52:0/462505299 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc8ac0679c0 0x7fc8ac069e80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009fd0 tx=0x7fc8b000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fd9823fa640 1 --2- 172.21.15.52:0/80853820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd97c06f4e0 0x7fd97c15bf10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c7c0 tx=0x7fd97400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fd972ffd640 1 -- 172.21.15.52:0/80853820 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974003ba0 con 0x7fd97c06f4e0 2024-05-04T12:18:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fd972ffd640 1 -- 172.21.15.52:0/80853820 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd974003d40 con 0x7fd97c06f4e0 2024-05-04T12:18:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fd972ffd640 1 -- 172.21.15.52:0/80853820 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97400aa90 con 0x7fd97c06f4e0 2024-05-04T12:18:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fd98465e640 1 -- 172.21.15.52:0/80853820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97c15dc90 con 0x7fd97c06f4e0 2024-05-04T12:18:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fd98465e640 1 -- 172.21.15.52:0/80853820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97c15e250 con 0x7fd97c06f4e0 2024-05-04T12:18:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fd98465e640 1 -- 172.21.15.52:0/80853820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd97c062e50 con 0x7fd97c06f4e0 2024-05-04T12:18:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fe1eaffd640 1 --2- 172.21.15.52:0/1001136053 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe1dc0678c0 0x7fe1dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fe1eaffd640 1 -- 172.21.15.52:0/1001136053 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe1f00856f0 con 0x7fe1fc072280 2024-05-04T12:18:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fe1f9d9b640 1 --2- 172.21.15.52:0/1001136053 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe1dc0678c0 0x7fe1dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fe1eaffd640 1 --2- 172.21.15.52:0/1001136053 >> [v2:172.21.15.52:6800/4058473938,v1:172.21.15.52:6801/4058473938] conn(0x7fe1dc06fae0 0x7fe1dc071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fe1eaffd640 1 -- 172.21.15.52:0/1001136053 --> [v2:172.21.15.52:6800/4058473938,v1:172.21.15.52:6801/4058473938] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe1dc0725b0 con 0x7fe1dc06fae0 2024-05-04T12:18:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fe1eaffd640 1 -- 172.21.15.52:0/1001136053 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe1f0085ae0 con 0x7fe1fc072280 2024-05-04T12:18:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fe1fad9d640 1 --2- 172.21.15.52:0/1001136053 >> [v2:172.21.15.52:6800/4058473938,v1:172.21.15.52:6801/4058473938] conn(0x7fe1dc06fae0 0x7fe1dc071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fe1f9d9b640 1 --2- 172.21.15.52:0/1001136053 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe1dc0678c0 0x7fe1dc069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0b2000 tx=0x7fe1ec0aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fe1fad9d640 1 --2- 172.21.15.52:0/1001136053 >> [v2:172.21.15.52:6800/4058473938,v1:172.21.15.52:6801/4058473938] conn(0x7fe1dc06fae0 0x7fe1dc071f00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7f623f7c6640 1 -- 172.21.15.52:0/2741192402 shutdown_connections 2024-05-04T12:18:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7f623f7c6640 1 -- 172.21.15.52:0/2741192402 wait complete. 2024-05-04T12:18:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7f623f7c6640 1 Processor -- start 2024-05-04T12:18:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7f93319f7640 1 --2- 172.21.15.52:0/2499366432 >> [v2:172.21.15.111:6803/1318681540,v1:172.21.15.111:6807/1318681540] conn(0x7f931006fa60 0x7f9310071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7f93319f7640 1 --2- 172.21.15.52:0/2499366432 >> [v2:172.21.15.111:6803/1318681540,v1:172.21.15.111:6807/1318681540] conn(0x7f931006fa60 0x7f9310071e80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc9d37fe640 1 -- 172.21.15.52:0/3968926859 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc9e00c4410 con 0x7fc9e4033e40 2024-05-04T12:18:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc9f100a640 1 -- 172.21.15.52:0/3968926859 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc9cc001040 con 0x7fc9e4033e40 2024-05-04T12:18:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc9d37fe640 1 --2- 172.21.15.52:0/3968926859 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc9d8067910 0x7fc9d8069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc9d37fe640 1 -- 172.21.15.52:0/3968926859 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc9e0122cd0 con 0x7fc9e4033e40 2024-05-04T12:18:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc9d37fe640 1 --2- 172.21.15.52:0/3968926859 >> [v2:172.21.15.111:6800/1985934042,v1:172.21.15.111:6801/1985934042] conn(0x7fc9d806fab0 0x7fc9d8071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc9d37fe640 1 -- 172.21.15.52:0/3968926859 --> [v2:172.21.15.111:6800/1985934042,v1:172.21.15.111:6801/1985934042] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc9e00a9e50 con 0x7fc9d806fab0 2024-05-04T12:18:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc9ead9d640 1 --2- 172.21.15.52:0/3968926859 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc9d8067910 0x7fc9d8069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc9eb59e640 1 --2- 172.21.15.52:0/3968926859 >> [v2:172.21.15.111:6800/1985934042,v1:172.21.15.111:6801/1985934042] conn(0x7fc9d806fab0 0x7fc9d8071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc9d37fe640 1 -- 172.21.15.52:0/3968926859 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc9e00ef520 con 0x7fc9e4033e40 2024-05-04T12:18:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc9ead9d640 1 --2- 172.21.15.52:0/3968926859 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc9d8067910 0x7fc9d8069dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec074830 tx=0x7fc9ec062c40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff09ad9d640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff09c139770 0x7ff09c139bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff09a59c640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff09c145730 0x7ff09c15ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff09a59c640 1 -- 172.21.15.52:0/279597511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff09c139770 msgr2=0x7ff09c139bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff09a59c640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff09c139770 0x7ff09c139bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff09a59c640 1 -- 172.21.15.52:0/279597511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c145b10 msgr2=0x7ff09c132d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff09a59c640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c145b10 0x7ff09c132d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff09a59c640 1 -- 172.21.15.52:0/279597511 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c15bd50 con 0x7ff09c145730 2024-05-04T12:18:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff099d9b640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c145b10 0x7ff09c132d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:18:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7ff09a59c640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff09c145730 0x7ff09c15ca80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c400 tx=0x7ff08c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc9eb59e640 1 --2- 172.21.15.52:0/3968926859 >> [v2:172.21.15.111:6800/1985934042,v1:172.21.15.111:6801/1985934042] conn(0x7fc9d806fab0 0x7fc9d8071ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fe1eaffd640 1 -- 172.21.15.52:0/1001136053 <== osd.2 v2:172.21.15.52:6800/4058473938 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe1dc0725b0 con 0x7fe1dc06fae0 2024-05-04T12:18:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7fc8a77fe640 1 -- 172.21.15.52:0/462505299 <== osd.0 v2:172.21.15.52:6808/2539299307 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc8c006e0f0 con 0x7fc8ac06fbe0 2024-05-04T12:18:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7f623f7c6640 1 -- start start 2024-05-04T12:18:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.558+0000 7f69dd7fa640 1 -- 172.21.15.52:0/3323135127 <== osd.4 v2:172.21.15.111:6823/2919151955 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f69cc001d40 con 0x7f69c406fbb0 2024-05-04T12:18:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623f7c6640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f623806e4c0 0x7f623814bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623f7c6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62380b28e0 0x7f6238150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623f7c6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f623815a210 0x7f623814c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623f7c6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f623815a7f0 con 0x7f623815a210 2024-05-04T12:18:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623f7c6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f623815a670 con 0x7f62380b28e0 2024-05-04T12:18:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623f7c6640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f62380b08a0 con 0x7f623806e4c0 2024-05-04T12:18:28.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623dd63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f623815a210 0x7f623814c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623dd63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f623815a210 0x7f623814c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53472/0 (socket says 172.21.15.52:53472) 2024-05-04T12:18:28.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623dd63640 1 -- 172.21.15.52:0/919265042 learned_addr learned my addr 172.21.15.52:0/919265042 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623d562640 1 --2- 172.21.15.52:0/919265042 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f623806e4c0 0x7f623814bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623dd63640 1 -- 172.21.15.52:0/919265042 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f623806e4c0 msgr2=0x7f623814bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623dd63640 1 --2- 172.21.15.52:0/919265042 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f623806e4c0 0x7f623814bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623dd63640 1 -- 172.21.15.52:0/919265042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62380b28e0 msgr2=0x7f6238150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623dd63640 1 --2- 172.21.15.52:0/919265042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62380b28e0 0x7f6238150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623dd63640 1 -- 172.21.15.52:0/919265042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6238151230 con 0x7f623815a210 2024-05-04T12:18:28.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623d562640 1 --2- 172.21.15.52:0/919265042 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f623806e4c0 0x7f623814bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:18:28.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623dd63640 1 --2- 172.21.15.52:0/919265042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f623815a210 0x7f623814c650 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f623000b9c0 tx=0x7f623000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f622dffb640 1 -- 172.21.15.52:0/919265042 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230019070 con 0x7f623815a210 2024-05-04T12:18:28.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f622dffb640 1 -- 172.21.15.52:0/919265042 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f623000cba0 con 0x7f623815a210 2024-05-04T12:18:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623f7c6640 1 -- 172.21.15.52:0/919265042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623815f1d0 con 0x7f623815a210 2024-05-04T12:18:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623f7c6640 1 -- 172.21.15.52:0/919265042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f623815f790 con 0x7f623815a210 2024-05-04T12:18:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7f623f7c6640 1 -- 172.21.15.52:0/919265042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f623814c480 con 0x7f623815a210 2024-05-04T12:18:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7fd972ffd640 1 -- 172.21.15.52:0/80853820 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd97400ac30 con 0x7fd97c06f4e0 2024-05-04T12:18:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7fd972ffd640 1 --2- 172.21.15.52:0/80853820 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd960067910 0x7fd960069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7fd972ffd640 1 -- 172.21.15.52:0/80853820 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd974086500 con 0x7fd97c06f4e0 2024-05-04T12:18:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7fd972ffd640 1 --2- 172.21.15.52:0/80853820 >> [v2:172.21.15.52:6816/1070276451,v1:172.21.15.52:6817/1070276451] conn(0x7fd96006fb30 0x7fd960071f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7fd972ffd640 1 -- 172.21.15.52:0/80853820 --> [v2:172.21.15.52:6816/1070276451,v1:172.21.15.52:6817/1070276451] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd97400bf30 con 0x7fd96006fb30 2024-05-04T12:18:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7fd972ffd640 1 -- 172.21.15.52:0/80853820 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd9740869e0 con 0x7fd97c06f4e0 2024-05-04T12:18:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7fd981bf9640 1 --2- 172.21.15.52:0/80853820 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd960067910 0x7fd960069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.562+0000 7fd981bf9640 1 --2- 172.21.15.52:0/80853820 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd960067910 0x7fd960069dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0af000 tx=0x7fd96c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7ff08affd640 1 -- 172.21.15.52:0/279597511 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0074e0 con 0x7ff09c145730 2024-05-04T12:18:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7fc9d37fe640 1 -- 172.21.15.52:0/3968926859 <== osd.6 v2:172.21.15.111:6800/1985934042 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc9e00a9e50 con 0x7fc9d806fab0 2024-05-04T12:18:28.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7ff09b59e640 1 -- 172.21.15.52:0/279597511 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09c133640 con 0x7ff09c145730 2024-05-04T12:18:28.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7ff09b59e640 1 -- 172.21.15.52:0/279597511 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09c15c2a0 con 0x7ff09c145730 2024-05-04T12:18:28.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7ff08affd640 1 -- 172.21.15.52:0/279597511 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff08c0076a0 con 0x7ff09c145730 2024-05-04T12:18:28.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7ff08affd640 1 -- 172.21.15.52:0/279597511 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c004a30 con 0x7ff09c145730 2024-05-04T12:18:28.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7ff09b59e640 1 -- 172.21.15.52:0/279597511 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff09c15c680 con 0x7ff09c145730 2024-05-04T12:18:28.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7f622dffb640 1 -- 172.21.15.52:0/919265042 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230014440 con 0x7f623815a210 2024-05-04T12:18:28.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7f622dffb640 1 -- 172.21.15.52:0/919265042 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6230007370 con 0x7f623815a210 2024-05-04T12:18:28.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7f622dffb640 1 --2- 172.21.15.52:0/919265042 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f621c0679f0 0x7f621c069eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7f622dffb640 1 -- 172.21.15.52:0/919265042 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6230085c60 con 0x7f623815a210 2024-05-04T12:18:28.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7f623d562640 1 --2- 172.21.15.52:0/919265042 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f621c0679f0 0x7f621c069eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7f9322ffd640 1 -- 172.21.15.52:0/2499366432 <== osd.5 v2:172.21.15.111:6803/1318681540 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9310072530 con 0x7f931006fa60 2024-05-04T12:18:28.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7fd982bfb640 1 --2- 172.21.15.52:0/80853820 >> [v2:172.21.15.52:6816/1070276451,v1:172.21.15.52:6817/1070276451] conn(0x7fd96006fb30 0x7fd960071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7f622dffb640 1 --2- 172.21.15.52:0/919265042 >> [v2:172.21.15.52:6824/2130169459,v1:172.21.15.52:6825/2130169459] conn(0x7f621c06fc10 0x7f621c072030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7f622dffb640 1 -- 172.21.15.52:0/919265042 --> [v2:172.21.15.52:6824/2130169459,v1:172.21.15.52:6825/2130169459] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f623000eea0 con 0x7f621c06fc10 2024-05-04T12:18:28.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7f622dffb640 1 -- 172.21.15.52:0/919265042 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6230085f80 con 0x7f623815a210 2024-05-04T12:18:28.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.566+0000 7f623cd61640 1 --2- 172.21.15.52:0/919265042 >> [v2:172.21.15.52:6824/2130169459,v1:172.21.15.52:6825/2130169459] conn(0x7f621c06fc10 0x7f621c072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.570+0000 7f623d562640 1 --2- 172.21.15.52:0/919265042 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f621c0679f0 0x7f621c069eb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f62340ad000 tx=0x7f62340a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.570+0000 7f623cd61640 1 --2- 172.21.15.52:0/919265042 >> [v2:172.21.15.52:6824/2130169459,v1:172.21.15.52:6825/2130169459] conn(0x7f621c06fc10 0x7f621c072030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.570+0000 7ff08affd640 1 -- 172.21.15.52:0/279597511 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff08c01a020 con 0x7ff09c145730 2024-05-04T12:18:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.570+0000 7ff08affd640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff07c067990 0x7ff07c069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.570+0000 7fd982bfb640 1 --2- 172.21.15.52:0/80853820 >> [v2:172.21.15.52:6816/1070276451,v1:172.21.15.52:6817/1070276451] conn(0x7fd96006fb30 0x7fd960071f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.570+0000 7f622dffb640 1 -- 172.21.15.52:0/919265042 <== osd.3 v2:172.21.15.52:6824/2130169459 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f623000eea0 con 0x7f621c06fc10 2024-05-04T12:18:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.570+0000 7ff099d9b640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff07c067990 0x7ff07c069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.570+0000 7ff08affd640 1 -- 172.21.15.52:0/279597511 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff08c0855d0 con 0x7ff09c145730 2024-05-04T12:18:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.570+0000 7ff08affd640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.111:6804/3267577455,v1:172.21.15.111:6806/3267577455] conn(0x7ff07c06fbb0 0x7ff07c071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.570+0000 7ff08affd640 1 -- 172.21.15.52:0/279597511 --> [v2:172.21.15.111:6804/3267577455,v1:172.21.15.111:6806/3267577455] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff07c072680 con 0x7ff07c06fbb0 2024-05-04T12:18:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.570+0000 7ff08affd640 1 -- 172.21.15.52:0/279597511 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff08c0859c0 con 0x7ff09c145730 2024-05-04T12:18:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.570+0000 7ff09ad9d640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.111:6804/3267577455,v1:172.21.15.111:6806/3267577455] conn(0x7ff07c06fbb0 0x7ff07c071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.570+0000 7fd972ffd640 1 -- 172.21.15.52:0/80853820 <== osd.1 v2:172.21.15.52:6816/1070276451 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd97400bf30 con 0x7fd96006fb30 2024-05-04T12:18:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.570+0000 7ff099d9b640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff07c067990 0x7ff07c069e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff090000c10 tx=0x7ff0900a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.578+0000 7ff09ad9d640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.111:6804/3267577455,v1:172.21.15.111:6806/3267577455] conn(0x7ff07c06fbb0 0x7ff07c071fd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.578+0000 7ff08affd640 1 -- 172.21.15.52:0/279597511 <== osd.7 v2:172.21.15.111:6804/3267577455 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff07c072680 con 0x7ff07c06fbb0 2024-05-04T12:18:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc8c559d640 1 -- 172.21.15.52:0/462505299 --> [v2:172.21.15.52:6808/2539299307,v1:172.21.15.52:6809/2539299307] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc8b80053a0 con 0x7fc8ac06fbe0 2024-05-04T12:18:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7f69e6f85640 1 -- 172.21.15.52:0/3323135127 --> [v2:172.21.15.111:6823/2919151955,v1:172.21.15.111:6825/2919151955] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f69d80053a0 con 0x7f69c406fbb0 2024-05-04T12:18:28.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc8a77fe640 1 -- 172.21.15.52:0/462505299 <== osd.0 v2:172.21.15.52:6808/2539299307 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc8b80053a0 con 0x7fc8ac06fbe0 2024-05-04T12:18:28.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7f69dd7fa640 1 -- 172.21.15.52:0/3323135127 <== osd.4 v2:172.21.15.111:6823/2919151955 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f69d80053a0 con 0x7f69c406fbb0 2024-05-04T12:18:28.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc8a57fa640 1 -- 172.21.15.52:0/462505299 >> [v2:172.21.15.52:6808/2539299307,v1:172.21.15.52:6809/2539299307] conn(0x7fc8ac06fbe0 msgr2=0x7fc8ac072000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc8a57fa640 1 --2- 172.21.15.52:0/462505299 >> [v2:172.21.15.52:6808/2539299307,v1:172.21.15.52:6809/2539299307] conn(0x7fc8ac06fbe0 0x7fc8ac072000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7f69caffd640 1 -- 172.21.15.52:0/3323135127 >> [v2:172.21.15.111:6823/2919151955,v1:172.21.15.111:6825/2919151955] conn(0x7f69c406fbb0 msgr2=0x7f69c4071fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7f69caffd640 1 --2- 172.21.15.52:0/3323135127 >> [v2:172.21.15.111:6823/2919151955,v1:172.21.15.111:6825/2919151955] conn(0x7f69c406fbb0 0x7f69c4071fd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc8a57fa640 1 -- 172.21.15.52:0/462505299 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc8ac0679c0 msgr2=0x7fc8ac069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc8a57fa640 1 --2- 172.21.15.52:0/462505299 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc8ac0679c0 0x7fc8ac069e80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009fd0 tx=0x7fc8b000b040 comp rx=0 tx=0).stop 2024-05-04T12:18:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc8a57fa640 1 -- 172.21.15.52:0/462505299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8b80df390 msgr2=0x7fc8b80cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc8a57fa640 1 --2- 172.21.15.52:0/462505299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8b80df390 0x7fc8b80cf260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc8c006a8f0 tx=0x7fc8c0093fc0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7f69caffd640 1 -- 172.21.15.52:0/3323135127 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f69c4067990 msgr2=0x7f69c4069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7f69caffd640 1 --2- 172.21.15.52:0/3323135127 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f69c4067990 0x7f69c4069e50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f69d4000f70 tx=0x7f69d4007240 comp rx=0 tx=0).stop 2024-05-04T12:18:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7f69caffd640 1 -- 172.21.15.52:0/3323135127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d804fa80 msgr2=0x7f69d80d7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7f69caffd640 1 --2- 172.21.15.52:0/3323135127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d804fa80 0x7f69d80d7cb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f69cc01b290 tx=0x7f69cc01cec0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc8bf7fe640 1 -- 172.21.15.52:0/462505299 reap_dead start 2024-05-04T12:18:28.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc8a57fa640 1 -- 172.21.15.52:0/462505299 shutdown_connections 2024-05-04T12:18:28.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc8a57fa640 1 -- 172.21.15.52:0/462505299 >> 172.21.15.52:0/462505299 conn(0x7fc8b8009fc0 msgr2=0x7fc8b804cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc8a57fa640 1 -- 172.21.15.52:0/462505299 shutdown_connections 2024-05-04T12:18:28.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc8a57fa640 1 -- 172.21.15.52:0/462505299 wait complete. 2024-05-04T12:18:28.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7f69e5522640 1 -- 172.21.15.52:0/3323135127 reap_dead start 2024-05-04T12:18:28.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7f69caffd640 1 -- 172.21.15.52:0/3323135127 shutdown_connections 2024-05-04T12:18:28.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7f69caffd640 1 -- 172.21.15.52:0/3323135127 >> 172.21.15.52:0/3323135127 conn(0x7f69d8009fc0 msgr2=0x7f69d80df770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7f69caffd640 1 -- 172.21.15.52:0/3323135127 shutdown_connections 2024-05-04T12:18:28.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7f69caffd640 1 -- 172.21.15.52:0/3323135127 wait complete. 2024-05-04T12:18:28.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc9f100a640 1 -- 172.21.15.52:0/3968926859 --> [v2:172.21.15.111:6800/1985934042,v1:172.21.15.111:6801/1985934042] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc9cc002d60 con 0x7fc9d806fab0 2024-05-04T12:18:28.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fe1e8ff9640 1 -- 172.21.15.52:0/1001136053 --> [v2:172.21.15.52:6800/4058473938,v1:172.21.15.52:6801/4058473938] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe1e0002de0 con 0x7fe1dc06fae0 2024-05-04T12:18:28.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.590+0000 7fc9d37fe640 1 -- 172.21.15.52:0/3968926859 <== osd.6 v2:172.21.15.111:6800/1985934042 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc9cc002d60 con 0x7fc9d806fab0 2024-05-04T12:18:28.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.594+0000 7fe1eaffd640 1 -- 172.21.15.52:0/1001136053 <== osd.2 v2:172.21.15.52:6800/4058473938 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe1e0002de0 con 0x7fe1dc06fae0 2024-05-04T12:18:28.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.594+0000 7fe1e8ff9640 1 -- 172.21.15.52:0/1001136053 >> [v2:172.21.15.52:6800/4058473938,v1:172.21.15.52:6801/4058473938] conn(0x7fe1dc06fae0 msgr2=0x7fe1dc071f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.594+0000 7fe1e8ff9640 1 --2- 172.21.15.52:0/1001136053 >> [v2:172.21.15.52:6800/4058473938,v1:172.21.15.52:6801/4058473938] conn(0x7fe1dc06fae0 0x7fe1dc071f00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.594+0000 7f933445c640 1 -- 172.21.15.52:0/2499366432 --> [v2:172.21.15.111:6803/1318681540,v1:172.21.15.111:6807/1318681540] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f932c154310 con 0x7f931006fa60 2024-05-04T12:18:28.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fe1e8ff9640 1 -- 172.21.15.52:0/1001136053 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe1dc0678c0 msgr2=0x7fe1dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fe1e8ff9640 1 --2- 172.21.15.52:0/1001136053 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe1dc0678c0 0x7fe1dc069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0b2000 tx=0x7fe1ec0aa070 comp rx=0 tx=0).stop 2024-05-04T12:18:28.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fe1e8ff9640 1 -- 172.21.15.52:0/1001136053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1fc072280 msgr2=0x7fe1fc1476e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fe1e8ff9640 1 --2- 172.21.15.52:0/1001136053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1fc072280 0x7fe1fc1476e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000c3c0 tx=0x7fe1f000c880 comp rx=0 tx=0).stop 2024-05-04T12:18:28.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fe1fad9d640 1 -- 172.21.15.52:0/1001136053 reap_dead start 2024-05-04T12:18:28.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fe1e8ff9640 1 -- 172.21.15.52:0/1001136053 shutdown_connections 2024-05-04T12:18:28.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fe1e8ff9640 1 -- 172.21.15.52:0/1001136053 >> 172.21.15.52:0/1001136053 conn(0x7fe1fc073180 msgr2=0x7fe1fc0b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fc9f100a640 1 -- 172.21.15.52:0/3968926859 >> [v2:172.21.15.111:6800/1985934042,v1:172.21.15.111:6801/1985934042] conn(0x7fc9d806fab0 msgr2=0x7fc9d8071ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fc9f100a640 1 --2- 172.21.15.52:0/3968926859 >> [v2:172.21.15.111:6800/1985934042,v1:172.21.15.111:6801/1985934042] conn(0x7fc9d806fab0 0x7fc9d8071ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fc9f100a640 1 -- 172.21.15.52:0/3968926859 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc9d8067910 msgr2=0x7fc9d8069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fc9f100a640 1 --2- 172.21.15.52:0/3968926859 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc9d8067910 0x7fc9d8069dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec074830 tx=0x7fc9ec062c40 comp rx=0 tx=0).stop 2024-05-04T12:18:28.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fc9f100a640 1 -- 172.21.15.52:0/3968926859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9e4033e40 msgr2=0x7fc9e40dc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fc9f100a640 1 --2- 172.21.15.52:0/3968926859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9e4033e40 0x7fc9e40dc2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00af000 tx=0x7fc9e001ba70 comp rx=0 tx=0).stop 2024-05-04T12:18:28.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fe1e8ff9640 1 -- 172.21.15.52:0/1001136053 shutdown_connections 2024-05-04T12:18:28.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fc9eb59e640 1 -- 172.21.15.52:0/3968926859 reap_dead start 2024-05-04T12:18:28.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fe1e8ff9640 1 -- 172.21.15.52:0/1001136053 wait complete. 2024-05-04T12:18:28.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fc9f100a640 1 -- 172.21.15.52:0/3968926859 shutdown_connections 2024-05-04T12:18:28.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fc9f100a640 1 -- 172.21.15.52:0/3968926859 >> 172.21.15.52:0/3968926859 conn(0x7fc9e4009fc0 msgr2=0x7fc9e40062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fc9f100a640 1 -- 172.21.15.52:0/3968926859 shutdown_connections 2024-05-04T12:18:28.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fc9f100a640 1 -- 172.21.15.52:0/3968926859 wait complete. 2024-05-04T12:18:28.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fd98465e640 1 -- 172.21.15.52:0/80853820 --> [v2:172.21.15.52:6816/1070276451,v1:172.21.15.52:6817/1070276451] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd97c075320 con 0x7fd96006fb30 2024-05-04T12:18:28.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fd972ffd640 1 -- 172.21.15.52:0/80853820 <== osd.1 v2:172.21.15.52:6816/1070276451 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd97c075320 con 0x7fd96006fb30 2024-05-04T12:18:28.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7f9322ffd640 1 -- 172.21.15.52:0/2499366432 <== osd.5 v2:172.21.15.111:6803/1318681540 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f932c154310 con 0x7f931006fa60 2024-05-04T12:18:28.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fd98465e640 1 -- 172.21.15.52:0/80853820 >> [v2:172.21.15.52:6816/1070276451,v1:172.21.15.52:6817/1070276451] conn(0x7fd96006fb30 msgr2=0x7fd960071f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fd98465e640 1 --2- 172.21.15.52:0/80853820 >> [v2:172.21.15.52:6816/1070276451,v1:172.21.15.52:6817/1070276451] conn(0x7fd96006fb30 0x7fd960071f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fd98465e640 1 -- 172.21.15.52:0/80853820 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd960067910 msgr2=0x7fd960069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fd98465e640 1 --2- 172.21.15.52:0/80853820 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd960067910 0x7fd960069dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0af000 tx=0x7fd96c0a7000 comp rx=0 tx=0).stop 2024-05-04T12:18:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7f933445c640 1 -- 172.21.15.52:0/2499366432 >> [v2:172.21.15.111:6803/1318681540,v1:172.21.15.111:6807/1318681540] conn(0x7f931006fa60 msgr2=0x7f9310071e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7f933445c640 1 --2- 172.21.15.52:0/2499366432 >> [v2:172.21.15.111:6803/1318681540,v1:172.21.15.111:6807/1318681540] conn(0x7f931006fa60 0x7f9310071e80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7f933445c640 1 -- 172.21.15.52:0/2499366432 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f93100678c0 msgr2=0x7f9310069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7f933445c640 1 --2- 172.21.15.52:0/2499366432 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f93100678c0 0x7f9310069d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9328000c10 tx=0x7f93280b5040 comp rx=0 tx=0).stop 2024-05-04T12:18:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7f933445c640 1 -- 172.21.15.52:0/2499366432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932c0d03c0 msgr2=0x7f932c06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7f933445c640 1 --2- 172.21.15.52:0/2499366432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932c0d03c0 0x7f932c06d8d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f932c0676d0 tx=0x7f932400ce40 comp rx=0 tx=0).stop 2024-05-04T12:18:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7ff09b59e640 1 -- 172.21.15.52:0/279597511 --> [v2:172.21.15.111:6804/3267577455,v1:172.21.15.111:6806/3267577455] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff09c062e50 con 0x7ff07c06fbb0 2024-05-04T12:18:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7f93329f9640 1 -- 172.21.15.52:0/2499366432 reap_dead start 2024-05-04T12:18:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7f933445c640 1 -- 172.21.15.52:0/2499366432 shutdown_connections 2024-05-04T12:18:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7f933445c640 1 -- 172.21.15.52:0/2499366432 >> 172.21.15.52:0/2499366432 conn(0x7f932c0b2210 msgr2=0x7f932c0afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7f933445c640 1 -- 172.21.15.52:0/2499366432 shutdown_connections 2024-05-04T12:18:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7f933445c640 1 -- 172.21.15.52:0/2499366432 wait complete. 2024-05-04T12:18:28.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7ff08affd640 1 -- 172.21.15.52:0/279597511 <== osd.7 v2:172.21.15.111:6804/3267577455 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff09c062e50 con 0x7ff07c06fbb0 2024-05-04T12:18:28.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7f623f7c6640 1 -- 172.21.15.52:0/919265042 --> [v2:172.21.15.52:6824/2130169459,v1:172.21.15.52:6825/2130169459] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f623814cde0 con 0x7f621c06fc10 2024-05-04T12:18:28.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7ff09b59e640 1 -- 172.21.15.52:0/279597511 >> [v2:172.21.15.111:6804/3267577455,v1:172.21.15.111:6806/3267577455] conn(0x7ff07c06fbb0 msgr2=0x7ff07c071fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7ff09b59e640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.111:6804/3267577455,v1:172.21.15.111:6806/3267577455] conn(0x7ff07c06fbb0 0x7ff07c071fd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.598+0000 7fd98465e640 1 -- 172.21.15.52:0/80853820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd97c06f4e0 msgr2=0x7fd97c15bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7fd98465e640 1 --2- 172.21.15.52:0/80853820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd97c06f4e0 0x7fd97c15bf10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c7c0 tx=0x7fd97400cc80 comp rx=0 tx=0).stop 2024-05-04T12:18:28.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7fd982bfb640 1 -- 172.21.15.52:0/80853820 reap_dead start 2024-05-04T12:18:28.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7fd98465e640 1 -- 172.21.15.52:0/80853820 shutdown_connections 2024-05-04T12:18:28.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7fd98465e640 1 -- 172.21.15.52:0/80853820 >> 172.21.15.52:0/80853820 conn(0x7fd97c072d80 msgr2=0x7fd97c073160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7fd98465e640 1 -- 172.21.15.52:0/80853820 shutdown_connections 2024-05-04T12:18:28.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7fd98465e640 1 -- 172.21.15.52:0/80853820 wait complete. 2024-05-04T12:18:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7ff09b59e640 1 -- 172.21.15.52:0/279597511 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff07c067990 msgr2=0x7ff07c069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7ff09b59e640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff07c067990 0x7ff07c069e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff090000c10 tx=0x7ff0900a6000 comp rx=0 tx=0).stop 2024-05-04T12:18:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7ff09b59e640 1 -- 172.21.15.52:0/279597511 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff09c145730 msgr2=0x7ff09c15ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7ff09b59e640 1 --2- 172.21.15.52:0/279597511 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff09c145730 0x7ff09c15ca80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c400 tx=0x7ff08c00c8c0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7ff09ad9d640 1 -- 172.21.15.52:0/279597511 reap_dead start 2024-05-04T12:18:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7ff09b59e640 1 -- 172.21.15.52:0/279597511 shutdown_connections 2024-05-04T12:18:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7ff09b59e640 1 -- 172.21.15.52:0/279597511 >> 172.21.15.52:0/279597511 conn(0x7ff09c067440 msgr2=0x7ff09c1310c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7ff09b59e640 1 -- 172.21.15.52:0/279597511 shutdown_connections 2024-05-04T12:18:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.602+0000 7ff09b59e640 1 -- 172.21.15.52:0/279597511 wait complete. 2024-05-04T12:18:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.606+0000 7f622dffb640 1 -- 172.21.15.52:0/919265042 <== osd.3 v2:172.21.15.52:6824/2130169459 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f623814cde0 con 0x7f621c06fc10 2024-05-04T12:18:28.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.614+0000 7f623f7c6640 1 -- 172.21.15.52:0/919265042 >> [v2:172.21.15.52:6824/2130169459,v1:172.21.15.52:6825/2130169459] conn(0x7f621c06fc10 msgr2=0x7f621c072030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.614+0000 7f623f7c6640 1 --2- 172.21.15.52:0/919265042 >> [v2:172.21.15.52:6824/2130169459,v1:172.21.15.52:6825/2130169459] conn(0x7f621c06fc10 0x7f621c072030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.614+0000 7f623f7c6640 1 -- 172.21.15.52:0/919265042 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f621c0679f0 msgr2=0x7f621c069eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.614+0000 7f623f7c6640 1 --2- 172.21.15.52:0/919265042 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f621c0679f0 0x7f621c069eb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f62340ad000 tx=0x7f62340a6000 comp rx=0 tx=0).stop 2024-05-04T12:18:28.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.614+0000 7f623f7c6640 1 -- 172.21.15.52:0/919265042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f623815a210 msgr2=0x7f623814c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.614+0000 7f623f7c6640 1 --2- 172.21.15.52:0/919265042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f623815a210 0x7f623814c650 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f623000b9c0 tx=0x7f623000be80 comp rx=0 tx=0).stop 2024-05-04T12:18:28.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.614+0000 7f623dd63640 1 -- 172.21.15.52:0/919265042 reap_dead start 2024-05-04T12:18:28.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.614+0000 7f623f7c6640 1 -- 172.21.15.52:0/919265042 shutdown_connections 2024-05-04T12:18:28.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.614+0000 7f623f7c6640 1 -- 172.21.15.52:0/919265042 >> 172.21.15.52:0/919265042 conn(0x7f6238067070 msgr2=0x7f62381534b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.614+0000 7f623f7c6640 1 -- 172.21.15.52:0/919265042 shutdown_connections 2024-05-04T12:18:28.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.614+0000 7f623f7c6640 1 -- 172.21.15.52:0/919265042 wait complete. 2024-05-04T12:18:28.637 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-04T12:18:28.637 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-04T12:18:28.638 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-04T12:18:28.638 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-04T12:18:28.639 INFO:teuthology.orchestra.run.smithi052.stdout:55834574855 2024-05-04T12:18:28.639 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-04T12:18:28.640 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-04T12:18:28.640 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-04T12:18:28.641 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-04T12:18:28.642 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-04T12:18:28.642 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-04T12:18:28.643 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-04T12:18:28.644 INFO:teuthology.orchestra.run.smithi052.stdout:55834574855 2024-05-04T12:18:28.644 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-04T12:18:28.646 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-04T12:18:28.647 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-04T12:18:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.766+0000 7f14fb7db640 1 Processor -- start 2024-05-04T12:18:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.766+0000 7f14fb7db640 1 -- start start 2024-05-04T12:18:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.766+0000 7f14fb7db640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f14f41437b0 0x7f14f40b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.766+0000 7f14fb7db640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14f4143520 con 0x7f14f4143140 2024-05-04T12:18:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.766+0000 7f14fb7db640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14f4070be0 con 0x7f14f40b1270 2024-05-04T12:18:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.766+0000 7f14fb7db640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f14f406d400 con 0x7f14f41437b0 2024-05-04T12:18:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14f9577640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f14f4143140 0x7f14f406cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51540/0 (socket says 172.21.15.52:51540) 2024-05-04T12:18:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14f9577640 1 -- 172.21.15.52:0/3656000634 learned_addr learned my addr 172.21.15.52:0/3656000634 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3479533354 0 0) 0x7f14f4143520 con 0x7f14f4143140 2024-05-04T12:18:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14e0003530 con 0x7f14f4143140 2024-05-04T12:18:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14f8d76640 1 --2- 172.21.15.52:0/3656000634 >> v2:172.21.15.111:3300/0 conn(0x7f14f41437b0 0x7f14f40b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1969500959 0 0) 0x7f14f4070be0 con 0x7f14f40b1270 2024-05-04T12:18:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14f4143520 con 0x7f14f40b1270 2024-05-04T12:18:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 140717861 0 0) 0x7f14e0003530 con 0x7f14f4143140 2024-05-04T12:18:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 >> v2:172.21.15.111:3300/0 conn(0x7f14f41437b0 msgr2=0x7f14f40b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 --2- 172.21.15.52:0/3656000634 >> v2:172.21.15.111:3300/0 conn(0x7f14f41437b0 0x7f14f40b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 >> v1:172.21.15.111:6789/0 conn(0x7f14f40b1270 legacy=0x7f14f40710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f406d6e0 con 0x7f14f4143140 2024-05-04T12:18:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f14e4001680 con 0x7f14f4143140 2024-05-04T12:18:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 >> v1:172.21.15.52:6789/0 conn(0x7f14f4143140 legacy=0x7f14f406cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 --2- 172.21.15.52:0/3656000634 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14e0003c10 0x7f14e0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 --2- 172.21.15.52:0/3656000634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e0004810 0x7f14e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 --2- 172.21.15.52:0/3656000634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e00053d0 0x7f14e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14f406d400 con 0x7f14e00053d0 2024-05-04T12:18:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f14e000c290 con 0x7f14e0004810 2024-05-04T12:18:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f14e000c490 con 0x7f14e0003c10 2024-05-04T12:18:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14f9577640 1 --2- 172.21.15.52:0/3656000634 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14e0003c10 0x7f14e0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14f9d78640 1 --2- 172.21.15.52:0/3656000634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e00053d0 0x7f14e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14f9577640 1 -- 172.21.15.52:0/3656000634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e0004810 msgr2=0x7f14e0004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T12:18:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14f9577640 1 --2- 172.21.15.52:0/3656000634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e0004810 0x7f14e0004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14f9577640 1 -- 172.21.15.52:0/3656000634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e00053d0 msgr2=0x7f14e000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14f9577640 1 --2- 172.21.15.52:0/3656000634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e00053d0 0x7f14e000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14f9577640 1 -- 172.21.15.52:0/3656000634 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e0003530 con 0x7f14e0003c10 2024-05-04T12:18:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14f9d78640 1 -- 172.21.15.52:0/3656000634 reap_dead start 2024-05-04T12:18:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.770+0000 7f14f8d76640 1 --2- 172.21.15.52:0/3656000634 >> v2:172.21.15.111:3300/0 conn(0x7f14f41437b0 0x7f14f40b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:28.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.774+0000 7f14f9577640 1 --2- 172.21.15.52:0/3656000634 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14e0003c10 0x7f14e0004010 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f14e400c0e0 tx=0x7f14e401f2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b2ae5492dd0381 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.774+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4007c50 con 0x7f14e0003c10 2024-05-04T12:18:28.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.774+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e40ac070 con 0x7f14e0003c10 2024-05-04T12:18:28.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.774+0000 7f14ebfff640 1 -- 172.21.15.52:0/3656000634 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4005b20 con 0x7f14e0003c10 2024-05-04T12:18:28.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.774+0000 7f14fb7db640 1 -- 172.21.15.52:0/3656000634 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14e0003c10 msgr2=0x7f14e0004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.774+0000 7f14fb7db640 1 --2- 172.21.15.52:0/3656000634 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14e0003c10 0x7f14e0004010 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f14e400c0e0 tx=0x7f14e401f2d0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.774+0000 7f14fb7db640 1 -- 172.21.15.52:0/3656000634 shutdown_connections 2024-05-04T12:18:28.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.774+0000 7f14fb7db640 1 --2- 172.21.15.52:0/3656000634 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14e0003c10 0x7f14e0004010 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.774+0000 7f14fb7db640 1 -- 172.21.15.52:0/3656000634 >> 172.21.15.52:0/3656000634 conn(0x7f14f4072d80 msgr2=0x7f14f4073190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.774+0000 7f14fb7db640 1 -- 172.21.15.52:0/3656000634 shutdown_connections 2024-05-04T12:18:28.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14fb7db640 1 -- 172.21.15.52:0/3656000634 wait complete. 2024-05-04T12:18:28.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14fb7db640 1 Processor -- start 2024-05-04T12:18:28.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14fb7db640 1 -- start start 2024-05-04T12:18:28.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14fb7db640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e0003c10 0x7f14f40ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14fb7db640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f40ab480 0x7f14f40a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14fb7db640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14f40ab900 0x7f14f40aa460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14fb7db640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14f406cbd0 con 0x7f14f40ab480 2024-05-04T12:18:28.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14fb7db640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f14f406dca0 con 0x7f14e0003c10 2024-05-04T12:18:28.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14fb7db640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f14f406cd50 con 0x7f14f40ab900 2024-05-04T12:18:28.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14f8d76640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f40ab480 0x7f14f40a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14f8d76640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f40ab480 0x7f14f40a9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53492/0 (socket says 172.21.15.52:53492) 2024-05-04T12:18:28.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14f9577640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e0003c10 0x7f14f40ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14f9577640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e0003c10 0x7f14f40ad5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:57074/0 (socket says 172.21.15.52:57074) 2024-05-04T12:18:28.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14f9577640 1 -- 172.21.15.52:0/1523808215 learned_addr learned my addr 172.21.15.52:0/1523808215 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.778+0000 7f14f9d78640 1 --2- 172.21.15.52:0/1523808215 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14f40ab900 0x7f14f40aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.782+0000 7f14f9577640 1 -- 172.21.15.52:0/1523808215 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14f40ab900 msgr2=0x7f14f40aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.782+0000 7f14f9577640 1 --2- 172.21.15.52:0/1523808215 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14f40ab900 0x7f14f40aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.782+0000 7f14f9577640 1 -- 172.21.15.52:0/1523808215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f40ab480 msgr2=0x7f14f40a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.782+0000 7f14f9577640 1 --2- 172.21.15.52:0/1523808215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f40ab480 0x7f14f40a9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.782+0000 7f14f9577640 1 -- 172.21.15.52:0/1523808215 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f416b700 con 0x7f14e0003c10 2024-05-04T12:18:28.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.782+0000 7f14f9577640 1 --2- 172.21.15.52:0/1523808215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e0003c10 0x7f14f40ad5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14e401f100 tx=0x7f14e0005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.782+0000 7f14e9ffb640 1 -- 172.21.15.52:0/1523808215 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4007c50 con 0x7f14e0003c10 2024-05-04T12:18:28.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.782+0000 7f14fb7db640 1 -- 172.21.15.52:0/1523808215 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f416b9e0 con 0x7f14e0003c10 2024-05-04T12:18:28.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.782+0000 7f14fb7db640 1 -- 172.21.15.52:0/1523808215 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f416bfa0 con 0x7f14e0003c10 2024-05-04T12:18:28.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.782+0000 7f14e9ffb640 1 -- 172.21.15.52:0/1523808215 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e4001480 con 0x7f14e0003c10 2024-05-04T12:18:28.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.782+0000 7f14e9ffb640 1 -- 172.21.15.52:0/1523808215 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4007340 con 0x7f14e0003c10 2024-05-04T12:18:28.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.782+0000 7f14fb7db640 1 -- 172.21.15.52:0/1523808215 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d80052a0 con 0x7f14e0003c10 2024-05-04T12:18:28.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.782+0000 7f14e9ffb640 1 -- 172.21.15.52:0/1523808215 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f14e40074e0 con 0x7f14e0003c10 2024-05-04T12:18:28.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.786+0000 7f14e9ffb640 1 --2- 172.21.15.52:0/1523808215 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f14e0069560 0x7f14e006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.786+0000 7f14e9ffb640 1 -- 172.21.15.52:0/1523808215 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f14e4122db0 con 0x7f14e0003c10 2024-05-04T12:18:28.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.786+0000 7f14f8d76640 1 --2- 172.21.15.52:0/1523808215 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f14e0069560 0x7f14e006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.786+0000 7f14f8d76640 1 --2- 172.21.15.52:0/1523808215 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f14e0069560 0x7f14e006ba20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f14f4159a80 tx=0x7f14f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f63f6d640 1 Processor -- start 2024-05-04T12:18:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f63f6d640 1 -- start start 2024-05-04T12:18:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0a1b59e640 1 Processor -- start 2024-05-04T12:18:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f7655d45640 1 Processor -- start 2024-05-04T12:18:28.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0a1b59e640 1 -- start start 2024-05-04T12:18:28.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f63f6d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0f5c15fd00 0x7f0f5c0b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f63f6d640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0f5c160190 0x7f0f5c0b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f63f6d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f5c160c30 con 0x7f0f5c159ba0 2024-05-04T12:18:28.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f63f6d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f0f5c071760 con 0x7f0f5c15fd00 2024-05-04T12:18:28.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f63f6d640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5c160670 con 0x7f0f5c160190 2024-05-04T12:18:28.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f61508640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0f5c15fd00 0x7f0f5c0b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f61508640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0f5c15fd00 0x7f0f5c0b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:57090/0 (socket says 172.21.15.52:57090) 2024-05-04T12:18:28.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f61508640 1 -- 172.21.15.52:0/2421166628 learned_addr learned my addr 172.21.15.52:0/2421166628 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f61d09640 1 --1- 172.21.15.52:0/2421166628 >> v1:172.21.15.111:6789/0 conn(0x7f0f5c159ba0 0x7f0f5c071050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.52:59632/0 (socket says 172.21.15.52:59632) 2024-05-04T12:18:28.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0a1b59e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0a1c15e080 0x7f0a1c0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0a1b59e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0a1c0b2de0 0x7f0a1c0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0a1b59e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a1c0ae510 con 0x7f0a1c0cc880 2024-05-04T12:18:28.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0a1b59e640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f0a1c0ae2e0 con 0x7f0a1c15e080 2024-05-04T12:18:28.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0a1b59e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f0a1c0ae7f0 con 0x7f0a1c0b2de0 2024-05-04T12:18:28.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0a1a59c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0a1c15e080 0x7f0a1c0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0a1a59c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0a1c15e080 0x7f0a1c0adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53496/0 (socket says 172.21.15.52:53496) 2024-05-04T12:18:28.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0a1a59c640 1 -- 172.21.15.52:0/4168897271 learned_addr learned my addr 172.21.15.52:0/4168897271 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f6250a640 1 --2- 172.21.15.52:0/2421166628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0f5c160190 0x7f0f5c0b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0a1a59c640 1 -- 172.21.15.52:0/4168897271 >> v2:172.21.15.111:3300/0 conn(0x7f0a1c0b2de0 msgr2=0x7f0a1c0cee80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T12:18:28.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0a1a59c640 1 --2- 172.21.15.52:0/4168897271 >> v2:172.21.15.111:3300/0 conn(0x7f0a1c0b2de0 0x7f0a1c0cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0a1a59c640 1 -- 172.21.15.52:0/4168897271 >> v1:172.21.15.111:6789/0 conn(0x7f0a1c0cc880 legacy=0x7f0a1c0b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0a1a59c640 1 -- 172.21.15.52:0/4168897271 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c0aead0 con 0x7f0a1c15e080 2024-05-04T12:18:28.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f7655d45640 1 -- start start 2024-05-04T12:18:28.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f7655d45640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76500cdae0 0x7f765015ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f7655d45640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f76500cdec0 0x7f765015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f7647fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f76500cdec0 0x7f765015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f7654d43640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76500cdae0 0x7f765015ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f7647fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f76500cdec0 0x7f765015aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53504/0 (socket says 172.21.15.52:53504) 2024-05-04T12:18:28.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f7654d43640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76500cdae0 0x7f765015ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.52:56094/0 (socket says 172.21.15.52:56094) 2024-05-04T12:18:28.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f7647fff640 1 -- 172.21.15.52:0/1658688065 learned_addr learned my addr 172.21.15.52:0/1658688065 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f7655d45640 1 -- 172.21.15.52:0/1658688065 --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765015b4f0 con 0x7f765015afe0 2024-05-04T12:18:28.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f7655d45640 1 -- 172.21.15.52:0/1658688065 --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f765015b7d0 con 0x7f76500cdec0 2024-05-04T12:18:28.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f7655d45640 1 -- 172.21.15.52:0/1658688065 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f765015e230 con 0x7f76500cdae0 2024-05-04T12:18:28.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f61508640 1 -- 172.21.15.52:0/2421166628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0f5c160190 msgr2=0x7f0f5c0b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f61508640 1 --2- 172.21.15.52:0/2421166628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0f5c160190 0x7f0f5c0b34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f61508640 1 -- 172.21.15.52:0/2421166628 >> v1:172.21.15.111:6789/0 conn(0x7f0f5c159ba0 legacy=0x7f0f5c071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.790+0000 7f0f61508640 1 -- 172.21.15.52:0/2421166628 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c160950 con 0x7f0f5c15fd00 2024-05-04T12:18:28.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f0a1a59c640 1 --2- 172.21.15.52:0/4168897271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c15e080 0x7f0a1c0adda0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0d10b0 tx=0x7f0a1401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118fc734267f84b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f0a1959a640 1 -- 172.21.15.52:0/4168897271 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a140a5850 con 0x7f0a1c15e080 2024-05-04T12:18:28.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f0f61508640 1 --2- 172.21.15.52:0/2421166628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f5c15fd00 0x7f0f5c0b2f60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009e50 tx=0x7f0f5401ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0ed25ccdee84b77 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f0a1959a640 1 -- 172.21.15.52:0/4168897271 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a140a59f0 con 0x7f0a1c15e080 2024-05-04T12:18:28.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f0a1959a640 1 -- 172.21.15.52:0/4168897271 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a140a5dc0 con 0x7f0a1c15e080 2024-05-04T12:18:28.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f0a1b59e640 1 -- 172.21.15.52:0/4168897271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c15e080 msgr2=0x7f0a1c0adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f0a1b59e640 1 --2- 172.21.15.52:0/4168897271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c15e080 0x7f0a1c0adda0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0d10b0 tx=0x7f0a1401c9d0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f0a1b59e640 1 -- 172.21.15.52:0/4168897271 shutdown_connections 2024-05-04T12:18:28.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f0a1b59e640 1 --2- 172.21.15.52:0/4168897271 >> v2:172.21.15.111:3300/0 conn(0x7f0a1c0b2de0 0x7f0a1c0cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f0a1b59e640 1 --2- 172.21.15.52:0/4168897271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c15e080 0x7f0a1c0adda0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f0a1b59e640 1 -- 172.21.15.52:0/4168897271 >> 172.21.15.52:0/4168897271 conn(0x7f0a1c067070 msgr2=0x7f0a1c067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f7654d43640 1 -- 172.21.15.52:0/1658688065 >> v2:172.21.15.52:3300/0 conn(0x7f76500cdec0 msgr2=0x7f765015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f7654d43640 1 --2- 172.21.15.52:0/1658688065 >> v2:172.21.15.52:3300/0 conn(0x7f76500cdec0 0x7f765015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f7654d43640 1 -- 172.21.15.52:0/1658688065 >> v1:172.21.15.111:6789/0 conn(0x7f765015afe0 legacy=0x7f765006e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f7654d43640 1 -- 172.21.15.52:0/1658688065 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765015e510 con 0x7f76500cdae0 2024-05-04T12:18:28.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f7647fff640 1 --2- 172.21.15.52:0/1658688065 >> v2:172.21.15.52:3300/0 conn(0x7f76500cdec0 0x7f765015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:18:28.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f0a1b59e640 1 -- 172.21.15.52:0/4168897271 shutdown_connections 2024-05-04T12:18:28.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f0a1b59e640 1 -- 172.21.15.52:0/4168897271 wait complete. 2024-05-04T12:18:28.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f7654d43640 1 --2- 172.21.15.52:0/1658688065 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76500cdae0 0x7f765015ea30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f764c0097c0 tx=0x7f764c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d26b3a03b1b6d73 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.794+0000 7f76477fe640 1 -- 172.21.15.52:0/1658688065 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c0b1070 con 0x7f76500cdae0 2024-05-04T12:18:28.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f76477fe640 1 -- 172.21.15.52:0/1658688065 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f764c0ac410 con 0x7f76500cdae0 2024-05-04T12:18:28.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f76477fe640 1 -- 172.21.15.52:0/1658688065 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c0ac7e0 con 0x7f76500cdae0 2024-05-04T12:18:28.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1b59e640 1 Processor -- start 2024-05-04T12:18:28.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1b59e640 1 -- start start 2024-05-04T12:18:28.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f7655d45640 1 -- 172.21.15.52:0/1658688065 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76500cdae0 msgr2=0x7f765015ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f7655d45640 1 --2- 172.21.15.52:0/1658688065 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76500cdae0 0x7f765015ea30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f764c0097c0 tx=0x7f764c01c4a0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1b59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a1c0b2de0 0x7f0a1c13ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1b59e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a1c0cc880 0x7f0a1c149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1b59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c15e080 0x7f0a1c14a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1b59e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c15e5e0 con 0x7f0a1c15e080 2024-05-04T12:18:28.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1b59e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c15e460 con 0x7f0a1c0b2de0 2024-05-04T12:18:28.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1b59e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f0a1c0cfbd0 con 0x7f0a1c0cc880 2024-05-04T12:18:28.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c15e080 0x7f0a1c14a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c15e080 0x7f0a1c14a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53514/0 (socket says 172.21.15.52:53514) 2024-05-04T12:18:28.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1ad9d640 1 -- 172.21.15.52:0/682620463 learned_addr learned my addr 172.21.15.52:0/682620463 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1a59c640 1 --2- 172.21.15.52:0/682620463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a1c0b2de0 0x7f0a1c13ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a19d9b640 1 --2- 172.21.15.52:0/682620463 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a1c0cc880 0x7f0a1c149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f14e9ffb640 1 -- 172.21.15.52:0/1523808215 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f14e4018fe0 con 0x7f14e0003c10 2024-05-04T12:18:28.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0f60d07640 1 -- 172.21.15.52:0/2421166628 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c15a190 con 0x7f0f5c15fd00 2024-05-04T12:18:28.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0f60d07640 1 -- 172.21.15.52:0/2421166628 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f540a5810 con 0x7f0f5c15fd00 2024-05-04T12:18:28.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0f60d07640 1 -- 172.21.15.52:0/2421166628 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f540a5be0 con 0x7f0f5c15fd00 2024-05-04T12:18:28.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1ad9d640 1 -- 172.21.15.52:0/682620463 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a1c0cc880 msgr2=0x7f0a1c149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1ad9d640 1 --2- 172.21.15.52:0/682620463 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a1c0cc880 0x7f0a1c149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1ad9d640 1 -- 172.21.15.52:0/682620463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a1c0b2de0 msgr2=0x7f0a1c13ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1ad9d640 1 --2- 172.21.15.52:0/682620463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a1c0b2de0 0x7f0a1c13ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.798+0000 7f0a1ad9d640 1 -- 172.21.15.52:0/682620463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c15d350 con 0x7f0a1c15e080 2024-05-04T12:18:28.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655d45640 1 -- 172.21.15.52:0/1658688065 shutdown_connections 2024-05-04T12:18:28.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655d45640 1 --2- 172.21.15.52:0/1658688065 >> v2:172.21.15.52:3300/0 conn(0x7f76500cdec0 0x7f765015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655d45640 1 --2- 172.21.15.52:0/1658688065 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76500cdae0 0x7f765015ea30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655d45640 1 -- 172.21.15.52:0/1658688065 >> 172.21.15.52:0/1658688065 conn(0x7f76500b2010 msgr2=0x7f76500b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 -- 172.21.15.52:0/2421166628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f5c15fd00 msgr2=0x7f0f5c0b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 --2- 172.21.15.52:0/2421166628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f5c15fd00 0x7f0f5c0b2f60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009e50 tx=0x7f0f5401ca30 comp rx=0 tx=0).stop 2024-05-04T12:18:28.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 -- 172.21.15.52:0/2421166628 shutdown_connections 2024-05-04T12:18:28.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 --2- 172.21.15.52:0/2421166628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0f5c160190 0x7f0f5c0b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 --2- 172.21.15.52:0/2421166628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f5c15fd00 0x7f0f5c0b2f60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655d45640 1 -- 172.21.15.52:0/1658688065 shutdown_connections 2024-05-04T12:18:28.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 -- 172.21.15.52:0/2421166628 >> 172.21.15.52:0/2421166628 conn(0x7f0f5c067070 msgr2=0x7f0f5c067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 -- 172.21.15.52:0/2421166628 shutdown_connections 2024-05-04T12:18:28.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 -- 172.21.15.52:0/2421166628 wait complete. 2024-05-04T12:18:28.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655d45640 1 -- 172.21.15.52:0/1658688065 wait complete. 2024-05-04T12:18:28.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655d45640 1 Processor -- start 2024-05-04T12:18:28.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 Processor -- start 2024-05-04T12:18:28.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655d45640 1 -- start start 2024-05-04T12:18:28.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655d45640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76500cdae0 0x7f765015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655d45640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76500cdec0 0x7f765014f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655d45640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f765015afe0 0x7f765014d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655d45640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f765015bc60 con 0x7f76500cdec0 2024-05-04T12:18:28.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655d45640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f765006ec50 con 0x7f765015afe0 2024-05-04T12:18:28.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655d45640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f765015bde0 con 0x7f76500cdae0 2024-05-04T12:18:28.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7654d43640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76500cdae0 0x7f765015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7654d43640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76500cdae0 0x7f765015d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.52:56124/0 (socket says 172.21.15.52:56124) 2024-05-04T12:18:28.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7654d43640 1 -- 172.21.15.52:0/2714785642 learned_addr learned my addr 172.21.15.52:0/2714785642 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655544640 1 --2- 172.21.15.52:0/2714785642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f765015afe0 0x7f765014d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655544640 1 -- 172.21.15.52:0/2714785642 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76500cdae0 msgr2=0x7f765015d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655544640 1 --2- 172.21.15.52:0/2714785642 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76500cdae0 0x7f765015d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655544640 1 -- 172.21.15.52:0/2714785642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76500cdec0 msgr2=0x7f765014f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655544640 1 --2- 172.21.15.52:0/2714785642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76500cdec0 0x7f765014f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655544640 1 -- 172.21.15.52:0/2714785642 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7650151290 con 0x7f765015afe0 2024-05-04T12:18:28.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 -- start start 2024-05-04T12:18:28.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0f5c159ba0 0x7f0f5c1585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f5c15fd00 0x7f0f5c204480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f5c160190 0x7f0f5c155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c0aa690 con 0x7f0f5c15fd00 2024-05-04T12:18:28.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c0aa510 con 0x7f0f5c160190 2024-05-04T12:18:28.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f63f6d640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5c0aa810 con 0x7f0f5c159ba0 2024-05-04T12:18:28.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f61d09640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0f5c159ba0 0x7f0f5c1585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f61d09640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0f5c159ba0 0x7f0f5c1585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.52:56126/0 (socket says 172.21.15.52:56126) 2024-05-04T12:18:28.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f61d09640 1 -- 172.21.15.52:0/3278330320 learned_addr learned my addr 172.21.15.52:0/3278330320 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f0f6250a640 1 --2- 172.21.15.52:0/3278330320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f5c160190 0x7f0f5c155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.802+0000 7f7655544640 1 --2- 172.21.15.52:0/2714785642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f765015afe0 0x7f765014d030 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f764800c760 tx=0x7f764800cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f0f6250a640 1 -- 172.21.15.52:0/3278330320 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0f5c159ba0 msgr2=0x7f0f5c1585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f0f6250a640 1 --2- 172.21.15.52:0/3278330320 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0f5c159ba0 0x7f0f5c1585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f0f6250a640 1 -- 172.21.15.52:0/3278330320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f5c15fd00 msgr2=0x7f0f5c204480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f0f6250a640 1 --2- 172.21.15.52:0/3278330320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f5c15fd00 0x7f0f5c204480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f0f6250a640 1 -- 172.21.15.52:0/3278330320 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c2049c0 con 0x7f0f5c160190 2024-05-04T12:18:28.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f0f6250a640 1 --2- 172.21.15.52:0/3278330320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f5c160190 0x7f0f5c155d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c1605a0 tx=0x7f0f5800c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f0f527fc640 1 -- 172.21.15.52:0/3278330320 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f58015070 con 0x7f0f5c160190 2024-05-04T12:18:28.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f76457fa640 1 -- 172.21.15.52:0/2714785642 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648016020 con 0x7f765015afe0 2024-05-04T12:18:28.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f7655d45640 1 -- 172.21.15.52:0/2714785642 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765014f860 con 0x7f765015afe0 2024-05-04T12:18:28.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f7655d45640 1 -- 172.21.15.52:0/2714785642 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f765014c830 con 0x7f765015afe0 2024-05-04T12:18:28.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f76457fa640 1 -- 172.21.15.52:0/2714785642 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7648006d60 con 0x7f765015afe0 2024-05-04T12:18:28.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f76457fa640 1 -- 172.21.15.52:0/2714785642 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648003bc0 con 0x7f765015afe0 2024-05-04T12:18:28.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f7655d45640 1 -- 172.21.15.52:0/2714785642 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7650062e50 con 0x7f765015afe0 2024-05-04T12:18:28.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f0f63f6d640 1 -- 172.21.15.52:0/3278330320 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5c157530 con 0x7f0f5c160190 2024-05-04T12:18:28.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f0f527fc640 1 -- 172.21.15.52:0/3278330320 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f58004220 con 0x7f0f5c160190 2024-05-04T12:18:28.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f0f527fc640 1 -- 172.21.15.52:0/3278330320 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f58003620 con 0x7f0f5c160190 2024-05-04T12:18:28.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f0f63f6d640 1 -- 172.21.15.52:0/3278330320 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5c157af0 con 0x7f0f5c160190 2024-05-04T12:18:28.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f0f63f6d640 1 -- 172.21.15.52:0/3278330320 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f5c062e50 con 0x7f0f5c160190 2024-05-04T12:18:28.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f76457fa640 1 -- 172.21.15.52:0/2714785642 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7648036840 con 0x7f765015afe0 2024-05-04T12:18:28.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.806+0000 7f76457fa640 1 --2- 172.21.15.52:0/2714785642 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f763c0678c0 0x7f763c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.810+0000 7f76457fa640 1 -- 172.21.15.52:0/2714785642 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7648085830 con 0x7f765015afe0 2024-05-04T12:18:28.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.810+0000 7f0a1ad9d640 1 --2- 172.21.15.52:0/682620463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c15e080 0x7f0a1c14a130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00b410 tx=0x7f0a0c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.810+0000 7f7654d43640 1 --2- 172.21.15.52:0/2714785642 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f763c0678c0 0x7f763c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.810+0000 7f0a0affd640 1 -- 172.21.15.52:0/682620463 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c015020 con 0x7f0a1c15e080 2024-05-04T12:18:28.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.810+0000 7f0a0affd640 1 -- 172.21.15.52:0/682620463 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a0c011850 con 0x7f0a1c15e080 2024-05-04T12:18:28.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.810+0000 7f0a0affd640 1 -- 172.21.15.52:0/682620463 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c011c40 con 0x7f0a1c15e080 2024-05-04T12:18:28.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.810+0000 7f7654d43640 1 --2- 172.21.15.52:0/2714785642 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f763c0678c0 0x7f763c069d80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f764c01ab50 tx=0x7f764c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.810+0000 7f0a1b59e640 1 -- 172.21.15.52:0/682620463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1c15c5d0 con 0x7f0a1c15e080 2024-05-04T12:18:28.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.810+0000 7f0a1b59e640 1 -- 172.21.15.52:0/682620463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a1c15cb50 con 0x7f0a1c15e080 2024-05-04T12:18:28.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.810+0000 7f0a1b59e640 1 -- 172.21.15.52:0/682620463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a1c062e50 con 0x7f0a1c15e080 2024-05-04T12:18:28.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.810+0000 7f0a0affd640 1 -- 172.21.15.52:0/682620463 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0a0c018020 con 0x7f0a1c15e080 2024-05-04T12:18:28.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.810+0000 7f0a0affd640 1 --2- 172.21.15.52:0/682620463 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f09fc0678c0 0x7f09fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.810+0000 7f0a0affd640 1 -- 172.21.15.52:0/682620463 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0a0c085a00 con 0x7f0a1c15e080 2024-05-04T12:18:28.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.810+0000 7f0a1a59c640 1 --2- 172.21.15.52:0/682620463 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f09fc0678c0 0x7f09fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.814+0000 7f0a1a59c640 1 --2- 172.21.15.52:0/682620463 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f09fc0678c0 0x7f09fc069d80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0a140ae000 tx=0x7f0a140a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.818+0000 7f76457fa640 1 -- 172.21.15.52:0/2714785642 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7648020020 con 0x7f765015afe0 2024-05-04T12:18:28.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.822+0000 7f0a0affd640 1 -- 172.21.15.52:0/682620463 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0a0c01f020 con 0x7f0a1c15e080 2024-05-04T12:18:28.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.826+0000 7f0f527fc640 1 -- 172.21.15.52:0/3278330320 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0f580037c0 con 0x7f0f5c160190 2024-05-04T12:18:28.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.826+0000 7f0f527fc640 1 --2- 172.21.15.52:0/3278330320 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0f400678c0 0x7f0f40069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.826+0000 7f0f527fc640 1 -- 172.21.15.52:0/3278330320 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0f58085d10 con 0x7f0f5c160190 2024-05-04T12:18:28.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.826+0000 7f0f61d09640 1 --2- 172.21.15.52:0/3278330320 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0f400678c0 0x7f0f40069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.826+0000 7f0f61d09640 1 --2- 172.21.15.52:0/3278330320 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0f400678c0 0x7f0f40069d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c000f70 tx=0x7f0f4c007ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.834+0000 7f0f527fc640 1 -- 172.21.15.52:0/3278330320 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0f580561e0 con 0x7f0f5c160190 2024-05-04T12:18:28.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.846+0000 7f3bbb66e640 1 Processor -- start 2024-05-04T12:18:28.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.846+0000 7f3bbb66e640 1 -- start start 2024-05-04T12:18:28.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3bbb66e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3bb415a210 0x7f3bb40b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3bbb66e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb40cd1d0 0x7f3bb406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3bbb66e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb40b12a0 con 0x7f3bb406df80 2024-05-04T12:18:28.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3bbb66e640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb406e920 con 0x7f3bb415a210 2024-05-04T12:18:28.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3bbb66e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb415e250 con 0x7f3bb40cd1d0 2024-05-04T12:18:28.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3bb940a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3bb415a210 0x7f3bb40b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3bb940a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3bb415a210 0x7f3bb40b0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53532/0 (socket says 172.21.15.52:53532) 2024-05-04T12:18:28.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3bb9c0b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb40cd1d0 0x7f3bb406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3bb8c09640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3bb406df80 0x7f3bb406cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.52:59668/0 (socket says 172.21.15.52:59668) 2024-05-04T12:18:28.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3bb8c09640 1 -- 172.21.15.52:0/3447992857 learned_addr learned my addr 172.21.15.52:0/3447992857 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:28.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3babfff640 1 -- 172.21.15.52:0/3447992857 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2992577245 0 0) 0x7f3bb40b12a0 con 0x7f3bb406df80 2024-05-04T12:18:28.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3bb9c0b640 1 -- 172.21.15.52:0/3447992857 >> v2:172.21.15.52:3300/0 conn(0x7f3bb415a210 msgr2=0x7f3bb40b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3bb9c0b640 1 --2- 172.21.15.52:0/3447992857 >> v2:172.21.15.52:3300/0 conn(0x7f3bb415a210 0x7f3bb40b0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3bb9c0b640 1 -- 172.21.15.52:0/3447992857 >> v1:172.21.15.111:6789/0 conn(0x7f3bb406df80 legacy=0x7f3bb406cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f3bb9c0b640 1 -- 172.21.15.52:0/3447992857 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb415e530 con 0x7f3bb40cd1d0 2024-05-04T12:18:28.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f25053d7640 1 Processor -- start 2024-05-04T12:18:28.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f25053d7640 1 -- start start 2024-05-04T12:18:28.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f25053d7640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25000cd990 0x7f250006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f25053d7640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2500071640 con 0x7f25000ccbc0 2024-05-04T12:18:28.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f25053d7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2500166ef0 con 0x7f25001625b0 2024-05-04T12:18:28.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f25053d7640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f2500070aa0 con 0x7f25000cd990 2024-05-04T12:18:28.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f24ff7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f25000ccbc0 0x7f25000ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51552/0 (socket says 172.21.15.52:51552) 2024-05-04T12:18:28.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.850+0000 7f24ff7fe640 1 -- 172.21.15.52:0/3980449775 learned_addr learned my addr 172.21.15.52:0/3980449775 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:28.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fe7fc640 1 --2- 172.21.15.52:0/3980449775 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25000cd990 0x7f250006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2489030257 0 0) 0x7f2500071640 con 0x7f25000ccbc0 2024-05-04T12:18:28.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24e8003530 con 0x7f25000ccbc0 2024-05-04T12:18:28.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f3bb9c0b640 1 --2- 172.21.15.52:0/3447992857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb40cd1d0 0x7f3bb406e3e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00097c0 tx=0x7f3bb001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf2fca28d846cfdd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1309896769 0 0) 0x7f2500166ef0 con 0x7f25001625b0 2024-05-04T12:18:28.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2500071640 con 0x7f25001625b0 2024-05-04T12:18:28.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 232050377 0 0) 0x7f24e8003530 con 0x7f25000ccbc0 2024-05-04T12:18:28.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25000cd990 msgr2=0x7f250006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 --2- 172.21.15.52:0/3980449775 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25000cd990 0x7f250006eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 >> v1:172.21.15.111:6789/0 conn(0x7f25001625b0 legacy=0x7f2500070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f3babfff640 1 -- 172.21.15.52:0/3447992857 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb00b0070 con 0x7f3bb40cd1d0 2024-05-04T12:18:28.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f3babfff640 1 -- 172.21.15.52:0/3447992857 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb00ab600 con 0x7f3bb40cd1d0 2024-05-04T12:18:28.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f3babfff640 1 -- 172.21.15.52:0/3447992857 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb00ab9d0 con 0x7f3bb40cd1d0 2024-05-04T12:18:28.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2500070730 con 0x7f25000ccbc0 2024-05-04T12:18:28.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f24f8001980 con 0x7f25000ccbc0 2024-05-04T12:18:28.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 >> v1:172.21.15.52:6789/0 conn(0x7f25000ccbc0 legacy=0x7f25000ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 --2- 172.21.15.52:0/3980449775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24e8003c50 0x7f24e8004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f3bbb66e640 1 -- 172.21.15.52:0/3447992857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb40cd1d0 msgr2=0x7f3bb406e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f3bbb66e640 1 --2- 172.21.15.52:0/3447992857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb40cd1d0 0x7f3bb406e3e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00097c0 tx=0x7f3bb001c4c0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f3bbb66e640 1 -- 172.21.15.52:0/3447992857 shutdown_connections 2024-05-04T12:18:28.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f3bbb66e640 1 --2- 172.21.15.52:0/3447992857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb40cd1d0 0x7f3bb406e3e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f3bbb66e640 1 --2- 172.21.15.52:0/3447992857 >> v2:172.21.15.52:3300/0 conn(0x7f3bb415a210 0x7f3bb40b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f3bbb66e640 1 -- 172.21.15.52:0/3447992857 >> 172.21.15.52:0/3447992857 conn(0x7f3bb4067070 msgr2=0x7f3bb4067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f3bbb66e640 1 -- 172.21.15.52:0/3447992857 shutdown_connections 2024-05-04T12:18:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24feffd640 1 --2- 172.21.15.52:0/3980449775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24e8003c50 0x7f24e8004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 --2- 172.21.15.52:0/3980449775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24e8004810 0x7f24e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 --2- 172.21.15.52:0/3980449775 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25000cd990 0x7f24e8005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2500070aa0 con 0x7f24e8003c50 2024-05-04T12:18:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fe7fc640 1 --2- 172.21.15.52:0/3980449775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24e8004810 0x7f24e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24ff7fe640 1 --2- 172.21.15.52:0/3980449775 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25000cd990 0x7f24e8005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f24e800bb60 con 0x7f24e8004810 2024-05-04T12:18:28.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f24e800bd60 con 0x7f25000cd990 2024-05-04T12:18:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.854+0000 7f3bbb66e640 1 -- 172.21.15.52:0/3447992857 wait complete. 2024-05-04T12:18:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bbb66e640 1 Processor -- start 2024-05-04T12:18:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bbb66e640 1 -- start start 2024-05-04T12:18:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bbb66e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb406df80 0x7f3bb414ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bbb66e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bb40cd1d0 0x7f3bb41687b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bbb66e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bb415a210 0x7f3bb4168cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bbb66e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb40b0a20 con 0x7f3bb40cd1d0 2024-05-04T12:18:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bbb66e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb40b08a0 con 0x7f3bb415a210 2024-05-04T12:18:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bbb66e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb415a5f0 con 0x7f3bb406df80 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bb940a640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb406df80 0x7f3bb414ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bb940a640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb406df80 0x7f3bb414ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.52:56152/0 (socket says 172.21.15.52:56152) 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bb940a640 1 -- 172.21.15.52:0/800672455 learned_addr learned my addr 172.21.15.52:0/800672455 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bb9c0b640 1 --2- 172.21.15.52:0/800672455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bb415a210 0x7f3bb4168cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bb940a640 1 -- 172.21.15.52:0/800672455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bb415a210 msgr2=0x7f3bb4168cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bb940a640 1 --2- 172.21.15.52:0/800672455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bb415a210 0x7f3bb4168cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bb940a640 1 -- 172.21.15.52:0/800672455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bb40cd1d0 msgr2=0x7f3bb41687b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bb940a640 1 --2- 172.21.15.52:0/800672455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bb40cd1d0 0x7f3bb41687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bb940a640 1 -- 172.21.15.52:0/800672455 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb414f160 con 0x7f3bb406df80 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bb940a640 1 --2- 172.21.15.52:0/800672455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb406df80 0x7f3bb414ec20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400d830 tx=0x7f3ba400dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3ba9ffb640 1 -- 172.21.15.52:0/800672455 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400b6c0 con 0x7f3bb406df80 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3ba9ffb640 1 -- 172.21.15.52:0/800672455 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba4019070 con 0x7f3bb406df80 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3ba9ffb640 1 -- 172.21.15.52:0/800672455 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400ba60 con 0x7f3bb406df80 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bbb66e640 1 -- 172.21.15.52:0/800672455 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb4169300 con 0x7f3bb406df80 2024-05-04T12:18:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f3bbb66e640 1 -- 172.21.15.52:0/800672455 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb4156510 con 0x7f3bb406df80 2024-05-04T12:18:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f24feffd640 1 -- 172.21.15.52:0/3980449775 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25000cd990 msgr2=0x7f24e8005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f24feffd640 1 --2- 172.21.15.52:0/3980449775 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25000cd990 0x7f24e8005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f24feffd640 1 -- 172.21.15.52:0/3980449775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24e8004810 msgr2=0x7f24e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f24feffd640 1 --2- 172.21.15.52:0/3980449775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24e8004810 0x7f24e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.858+0000 7f24feffd640 1 -- 172.21.15.52:0/3980449775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e8003530 con 0x7f24e8003c50 2024-05-04T12:18:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f24feffd640 1 --2- 172.21.15.52:0/3980449775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24e8003c50 0x7f24e8004030 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f24f0001800 tx=0x7f24f001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a5e8f7262c6964 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0001450 con 0x7f24e8003c50 2024-05-04T12:18:28.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f3bbb66e640 1 -- 172.21.15.52:0/800672455 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb4062e50 con 0x7f3bb406df80 2024-05-04T12:18:28.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f24fdffb640 1 -- 172.21.15.52:0/3980449775 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f00052f0 con 0x7f24e8003c50 2024-05-04T12:18:28.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f25053d7640 1 -- 172.21.15.52:0/3980449775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24e8003c50 msgr2=0x7f24e8004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f25053d7640 1 --2- 172.21.15.52:0/3980449775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24e8003c50 0x7f24e8004030 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f24f0001800 tx=0x7f24f001c4e0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f24ff7fe640 1 -- 172.21.15.52:0/3980449775 reap_dead start 2024-05-04T12:18:28.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f25053d7640 1 -- 172.21.15.52:0/3980449775 shutdown_connections 2024-05-04T12:18:28.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f25053d7640 1 -- 172.21.15.52:0/3980449775 >> 172.21.15.52:0/3980449775 conn(0x7f25000b14a0 msgr2=0x7f25000b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f3ba9ffb640 1 -- 172.21.15.52:0/800672455 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3ba401e460 con 0x7f3bb406df80 2024-05-04T12:18:28.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f25053d7640 1 -- 172.21.15.52:0/3980449775 shutdown_connections 2024-05-04T12:18:28.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f25053d7640 1 -- 172.21.15.52:0/3980449775 wait complete. 2024-05-04T12:18:28.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f3ba9ffb640 1 --2- 172.21.15.52:0/800672455 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f3ba0067a20 0x7f3ba0069ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f3ba9ffb640 1 -- 172.21.15.52:0/800672455 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3ba4086360 con 0x7f3bb406df80 2024-05-04T12:18:28.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f3bb8c09640 1 --2- 172.21.15.52:0/800672455 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f3ba0067a20 0x7f3ba0069ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.862+0000 7f3bb8c09640 1 --2- 172.21.15.52:0/800672455 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f3ba0067a20 0x7f3ba0069ee0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3bac000f90 tx=0x7f3bac007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f25053d7640 1 Processor -- start 2024-05-04T12:18:28.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f25053d7640 1 -- start start 2024-05-04T12:18:28.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f25053d7640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2500164cb0 0x7f25000cea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f25053d7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25000cefa0 0x7f25002046c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f24feffd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2500164cb0 0x7f25000cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f24feffd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2500164cb0 0x7f25000cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.52:56168/0 (socket says 172.21.15.52:56168) 2024-05-04T12:18:28.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f24fe7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25000cefa0 0x7f25002046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f24fe7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25000cefa0 0x7f25002046c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53552/0 (socket says 172.21.15.52:53552) 2024-05-04T12:18:28.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f24feffd640 1 -- 172.21.15.52:0/3645570885 learned_addr learned my addr 172.21.15.52:0/3645570885 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f25053d7640 1 --2- 172.21.15.52:0/3645570885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2500165090 0x7f2500204c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f25053d7640 1 -- 172.21.15.52:0/3645570885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f250016e060 con 0x7f25000cefa0 2024-05-04T12:18:28.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f25053d7640 1 -- 172.21.15.52:0/3645570885 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f250016dee0 con 0x7f2500165090 2024-05-04T12:18:28.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f25053d7640 1 -- 172.21.15.52:0/3645570885 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f250016e1e0 con 0x7f2500164cb0 2024-05-04T12:18:28.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f24ff7fe640 1 --2- 172.21.15.52:0/3645570885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2500165090 0x7f2500204c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f24fe7fc640 1 -- 172.21.15.52:0/3645570885 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2500164cb0 msgr2=0x7f25000cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f24fe7fc640 1 --2- 172.21.15.52:0/3645570885 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2500164cb0 0x7f25000cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f24fe7fc640 1 -- 172.21.15.52:0/3645570885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2500165090 msgr2=0x7f2500204c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f24fe7fc640 1 --2- 172.21.15.52:0/3645570885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2500165090 0x7f2500204c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.866+0000 7f24fe7fc640 1 -- 172.21.15.52:0/3645570885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25002051f0 con 0x7f25000cefa0 2024-05-04T12:18:28.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.870+0000 7f24fe7fc640 1 --2- 172.21.15.52:0/3645570885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25000cefa0 0x7f25002046c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f24f4011960 tx=0x7f24f4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.870+0000 7f24e77fe640 1 -- 172.21.15.52:0/3645570885 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4019070 con 0x7f25000cefa0 2024-05-04T12:18:28.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.870+0000 7f24e77fe640 1 -- 172.21.15.52:0/3645570885 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f4004300 con 0x7f25000cefa0 2024-05-04T12:18:28.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.870+0000 7f24e77fe640 1 -- 172.21.15.52:0/3645570885 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f400fa40 con 0x7f25000cefa0 2024-05-04T12:18:28.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.870+0000 7f25053d7640 1 -- 172.21.15.52:0/3645570885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25002054d0 con 0x7f25000cefa0 2024-05-04T12:18:28.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.870+0000 7f25053d7640 1 -- 172.21.15.52:0/3645570885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25000ce540 con 0x7f25000cefa0 2024-05-04T12:18:28.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.870+0000 7f25053d7640 1 -- 172.21.15.52:0/3645570885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e00052a0 con 0x7f25000cefa0 2024-05-04T12:18:28.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.870+0000 7f24e77fe640 1 -- 172.21.15.52:0/3645570885 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f24f401c070 con 0x7f25000cefa0 2024-05-04T12:18:28.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.878+0000 7f3ba9ffb640 1 -- 172.21.15.52:0/800672455 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3bb414f160 con 0x7f3bb406df80 2024-05-04T12:18:28.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.870+0000 7f24e77fe640 1 --2- 172.21.15.52:0/3645570885 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f24ec0678c0 0x7f24ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.878+0000 7f24e77fe640 1 -- 172.21.15.52:0/3645570885 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f24f4084730 con 0x7f25000cefa0 2024-05-04T12:18:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.878+0000 7f24feffd640 1 --2- 172.21.15.52:0/3645570885 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f24ec0678c0 0x7f24ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.882+0000 7f24feffd640 1 --2- 172.21.15.52:0/3645570885 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f24ec0678c0 0x7f24ec069d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24f001c310 tx=0x7f24f00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.882+0000 7f24e77fe640 1 -- 172.21.15.52:0/3645570885 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f24f40044b0 con 0x7f25000cefa0 2024-05-04T12:18:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.886+0000 7f5f62bf7640 1 Processor -- start 2024-05-04T12:18:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.886+0000 7f5f62bf7640 1 -- start start 2024-05-04T12:18:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.886+0000 7f5f62bf7640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5f5c14bd60 0x7f5f5c163640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.886+0000 7f5f62bf7640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f5c0ce3a0 con 0x7f5f5c0b2de0 2024-05-04T12:18:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.886+0000 7f5f62bf7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f5c0ccb50 con 0x7f5f5c0cd630 2024-05-04T12:18:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.886+0000 7f5f62bf7640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f5f5c0cce30 con 0x7f5f5c14bd60 2024-05-04T12:18:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.886+0000 7f5f623f6640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f5f5c0b2de0 0x7f5f5c070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51554/0 (socket says 172.21.15.52:51554) 2024-05-04T12:18:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.886+0000 7f5f623f6640 1 -- 172.21.15.52:0/822658629 learned_addr learned my addr 172.21.15.52:0/822658629 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.886+0000 7f5f61bf5640 1 --2- 172.21.15.52:0/822658629 >> v2:172.21.15.52:3300/0 conn(0x7f5f5c14bd60 0x7f5f5c163640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.886+0000 7f5f61bf5640 1 -- 172.21.15.52:0/822658629 >> v1:172.21.15.111:6789/0 conn(0x7f5f5c0cd630 legacy=0x7f5f5c0b26d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T12:18:28.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.886+0000 7f5f61bf5640 1 -- 172.21.15.52:0/822658629 >> v1:172.21.15.52:6789/0 conn(0x7f5f5c0b2de0 legacy=0x7f5f5c070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.886+0000 7f5f61bf5640 1 -- 172.21.15.52:0/822658629 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5c0cd110 con 0x7f5f5c14bd60 2024-05-04T12:18:28.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.890+0000 7f5f61bf5640 1 --2- 172.21.15.52:0/822658629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f5c14bd60 0x7f5f5c163640 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5f580097c0 tx=0x7f5f5801c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a134d9bea840a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.894+0000 7f5f60bf3640 1 -- 172.21.15.52:0/822658629 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f580b0070 con 0x7f5f5c14bd60 2024-05-04T12:18:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.894+0000 7f5f60bf3640 1 -- 172.21.15.52:0/822658629 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f580ab600 con 0x7f5f5c14bd60 2024-05-04T12:18:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.894+0000 7f5f62bf7640 1 -- 172.21.15.52:0/822658629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f5c14bd60 msgr2=0x7f5f5c163640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.894+0000 7f5f62bf7640 1 --2- 172.21.15.52:0/822658629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f5c14bd60 0x7f5f5c163640 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5f580097c0 tx=0x7f5f5801c2f0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.894+0000 7f5f62bf7640 1 -- 172.21.15.52:0/822658629 shutdown_connections 2024-05-04T12:18:28.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.894+0000 7f5f62bf7640 1 --2- 172.21.15.52:0/822658629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f5c14bd60 0x7f5f5c163640 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.894+0000 7f5f62bf7640 1 -- 172.21.15.52:0/822658629 >> 172.21.15.52:0/822658629 conn(0x7f5f5c067070 msgr2=0x7f5f5c067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.894+0000 7f5f62bf7640 1 -- 172.21.15.52:0/822658629 shutdown_connections 2024-05-04T12:18:28.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.894+0000 7f5f62bf7640 1 -- 172.21.15.52:0/822658629 wait complete. 2024-05-04T12:18:28.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f62bf7640 1 Processor -- start 2024-05-04T12:18:28.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f62bf7640 1 -- start start 2024-05-04T12:18:28.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f62bf7640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f5c0b2de0 0x7f5f5c071a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f62bf7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f5c0cd630 0x7f5f5c163300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f62bf7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f5c14bd60 0x7f5f5c163840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f62bf7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c14c140 con 0x7f5f5c0cd630 2024-05-04T12:18:28.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f62bf7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c0ce680 con 0x7f5f5c14bd60 2024-05-04T12:18:28.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f62bf7640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5f5c14c2c0 con 0x7f5f5c0b2de0 2024-05-04T12:18:28.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f613f4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f5c0cd630 0x7f5f5c163300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f613f4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f5c0cd630 0x7f5f5c163300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53566/0 (socket says 172.21.15.52:53566) 2024-05-04T12:18:28.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f613f4640 1 -- 172.21.15.52:0/3482405466 learned_addr learned my addr 172.21.15.52:0/3482405466 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f623f6640 1 --2- 172.21.15.52:0/3482405466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f5c14bd60 0x7f5f5c163840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f61bf5640 1 --2- 172.21.15.52:0/3482405466 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f5c0b2de0 0x7f5f5c071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f623f6640 1 -- 172.21.15.52:0/3482405466 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f5c0b2de0 msgr2=0x7f5f5c071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f623f6640 1 --2- 172.21.15.52:0/3482405466 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f5c0b2de0 0x7f5f5c071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f623f6640 1 -- 172.21.15.52:0/3482405466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f5c0cd630 msgr2=0x7f5f5c163300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.898+0000 7f5f623f6640 1 --2- 172.21.15.52:0/3482405466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f5c0cd630 0x7f5f5c163300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f5f623f6640 1 -- 172.21.15.52:0/3482405466 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5c15a970 con 0x7f5f5c14bd60 2024-05-04T12:18:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f5f623f6640 1 --2- 172.21.15.52:0/3482405466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f5c14bd60 0x7f5f5c163840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400e480 tx=0x7f5f5400e940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f5f527fc640 1 -- 172.21.15.52:0/3482405466 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f540127f0 con 0x7f5f5c14bd60 2024-05-04T12:18:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f5f62bf7640 1 -- 172.21.15.52:0/3482405466 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5c15ac50 con 0x7f5f5c14bd60 2024-05-04T12:18:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f5f62bf7640 1 -- 172.21.15.52:0/3482405466 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5c15b210 con 0x7f5f5c14bd60 2024-05-04T12:18:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f5f527fc640 1 -- 172.21.15.52:0/3482405466 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f54018070 con 0x7f5f5c14bd60 2024-05-04T12:18:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f5f527fc640 1 -- 172.21.15.52:0/3482405466 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54012b90 con 0x7f5f5c14bd60 2024-05-04T12:18:28.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f5f527fc640 1 -- 172.21.15.52:0/3482405466 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5f54013480 con 0x7f5f5c14bd60 2024-05-04T12:18:28.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f5f527fc640 1 --2- 172.21.15.52:0/3482405466 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5f48067a10 0x7f5f48069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.906+0000 7f5f61bf5640 1 --2- 172.21.15.52:0/3482405466 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5f48067a10 0x7f5f48069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f673a83b640 1 Processor -- start 2024-05-04T12:18:28.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f673a83b640 1 -- start start 2024-05-04T12:18:28.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f673a83b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6734070690 0x7f6734070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f673a83b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f673406f4e0 0x7f673406f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f673a83b640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67340725f0 con 0x7f673414bbf0 2024-05-04T12:18:28.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.906+0000 7f5f61bf5640 1 --2- 172.21.15.52:0/3482405466 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5f48067a10 0x7f5f48069ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5f580ad000 tx=0x7f5f580a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.906+0000 7f5f527fc640 1 -- 172.21.15.52:0/3482405466 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5f54088c80 con 0x7f5f5c14bd60 2024-05-04T12:18:28.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.906+0000 7f5f62bf7640 1 -- 172.21.15.52:0/3482405466 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f400052a0 con 0x7f5f5c14bd60 2024-05-04T12:18:28.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f673a83b640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f673406fe20 con 0x7f673406f4e0 2024-05-04T12:18:28.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.902+0000 7f673a83b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f673406e7d0 con 0x7f6734070690 2024-05-04T12:18:28.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.906+0000 7f6738dd8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f673406f4e0 0x7f673406f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.906+0000 7f6738dd8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f673406f4e0 0x7f673406f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53582/0 (socket says 172.21.15.52:53582) 2024-05-04T12:18:28.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.906+0000 7f6738dd8640 1 -- 172.21.15.52:0/352416947 learned_addr learned my addr 172.21.15.52:0/352416947 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.906+0000 7f6733fff640 1 --1- 172.21.15.52:0/352416947 >> v1:172.21.15.52:6789/0 conn(0x7f673414bbf0 0x7f67341634d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51568/0 (socket says 172.21.15.52:51568) 2024-05-04T12:18:28.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.906+0000 7f6738dd8640 1 -- 172.21.15.52:0/352416947 >> v2:172.21.15.111:3300/0 conn(0x7f6734070690 msgr2=0x7f6734070b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T12:18:28.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.906+0000 7f6738dd8640 1 --2- 172.21.15.52:0/352416947 >> v2:172.21.15.111:3300/0 conn(0x7f6734070690 0x7f6734070b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.906+0000 7f6738dd8640 1 -- 172.21.15.52:0/352416947 >> v1:172.21.15.52:6789/0 conn(0x7f673414bbf0 legacy=0x7f67341634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.906+0000 7f6738dd8640 1 -- 172.21.15.52:0/352416947 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673406d020 con 0x7f673406f4e0 2024-05-04T12:18:28.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.906+0000 7f6738dd8640 1 --2- 172.21.15.52:0/352416947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f673406f4e0 0x7f673406f8e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f67340b38a0 tx=0x7f672401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54882a441d35c51 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.910+0000 7f6732ffd640 1 -- 172.21.15.52:0/352416947 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67240ab430 con 0x7f673406f4e0 2024-05-04T12:18:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.910+0000 7f6732ffd640 1 -- 172.21.15.52:0/352416947 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67240b0070 con 0x7f673406f4e0 2024-05-04T12:18:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.910+0000 7f6732ffd640 1 -- 172.21.15.52:0/352416947 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67240ab7f0 con 0x7f673406f4e0 2024-05-04T12:18:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.910+0000 7f673a83b640 1 -- 172.21.15.52:0/352416947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f673406f4e0 msgr2=0x7f673406f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.910+0000 7f673a83b640 1 --2- 172.21.15.52:0/352416947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f673406f4e0 0x7f673406f8e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f67340b38a0 tx=0x7f672401c540 comp rx=0 tx=0).stop 2024-05-04T12:18:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.910+0000 7f673a83b640 1 -- 172.21.15.52:0/352416947 shutdown_connections 2024-05-04T12:18:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.910+0000 7f673a83b640 1 --2- 172.21.15.52:0/352416947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f673406f4e0 0x7f673406f8e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.910+0000 7f673a83b640 1 --2- 172.21.15.52:0/352416947 >> v2:172.21.15.111:3300/0 conn(0x7f6734070690 0x7f6734070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.910+0000 7f673a83b640 1 -- 172.21.15.52:0/352416947 >> 172.21.15.52:0/352416947 conn(0x7f6734072d80 msgr2=0x7f6734073190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.910+0000 7f673a83b640 1 -- 172.21.15.52:0/352416947 shutdown_connections 2024-05-04T12:18:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.910+0000 7f673a83b640 1 -- 172.21.15.52:0/352416947 wait complete. 2024-05-04T12:18:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.910+0000 7f673a83b640 1 Processor -- start 2024-05-04T12:18:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.910+0000 7f673a83b640 1 -- start start 2024-05-04T12:18:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f673a83b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f673406f4e0 0x7f67340cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f673a83b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6734070690 0x7f67340c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f673a83b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f673414bbf0 0x7f67340c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f673a83b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f673414c150 con 0x7f6734070690 2024-05-04T12:18:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f673a83b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f673414bfd0 con 0x7f673414bbf0 2024-05-04T12:18:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f673a83b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f673406cbd0 con 0x7f673406f4e0 2024-05-04T12:18:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f67337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6734070690 0x7f67340c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f6733fff640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f673406f4e0 0x7f67340cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f67337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6734070690 0x7f67340c9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53594/0 (socket says 172.21.15.52:53594) 2024-05-04T12:18:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f67337fe640 1 -- 172.21.15.52:0/62963428 learned_addr learned my addr 172.21.15.52:0/62963428 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f6738dd8640 1 --2- 172.21.15.52:0/62963428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f673414bbf0 0x7f67340c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f67337fe640 1 -- 172.21.15.52:0/62963428 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f673406f4e0 msgr2=0x7f67340cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f67337fe640 1 --2- 172.21.15.52:0/62963428 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f673406f4e0 0x7f67340cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f67337fe640 1 -- 172.21.15.52:0/62963428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f673414bbf0 msgr2=0x7f67340c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f67337fe640 1 --2- 172.21.15.52:0/62963428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f673414bbf0 0x7f67340c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f67337fe640 1 -- 172.21.15.52:0/62963428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67340cb710 con 0x7f6734070690 2024-05-04T12:18:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f67337fe640 1 --2- 172.21.15.52:0/62963428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6734070690 0x7f67340c9a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f672800b3f0 tx=0x7f672800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f6730ff9640 1 -- 172.21.15.52:0/62963428 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728015020 con 0x7f6734070690 2024-05-04T12:18:28.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f673a83b640 1 -- 172.21.15.52:0/62963428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6734140d50 con 0x7f6734070690 2024-05-04T12:18:28.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.914+0000 7f673a83b640 1 -- 172.21.15.52:0/62963428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67340cac30 con 0x7f6734070690 2024-05-04T12:18:28.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.918+0000 7f673a83b640 1 -- 172.21.15.52:0/62963428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6734062e50 con 0x7f6734070690 2024-05-04T12:18:28.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.918+0000 7f5f527fc640 1 -- 172.21.15.52:0/3482405466 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5f54059160 con 0x7f5f5c14bd60 2024-05-04T12:18:28.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.930+0000 7f6730ff9640 1 -- 172.21.15.52:0/62963428 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6728011850 con 0x7f6734070690 2024-05-04T12:18:28.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.930+0000 7f6730ff9640 1 -- 172.21.15.52:0/62963428 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728011c40 con 0x7f6734070690 2024-05-04T12:18:28.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.930+0000 7f6730ff9640 1 -- 172.21.15.52:0/62963428 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6728018020 con 0x7f6734070690 2024-05-04T12:18:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.930+0000 7f6730ff9640 1 --2- 172.21.15.52:0/62963428 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67200679c0 0x7f6720069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.930+0000 7f6733fff640 1 --2- 172.21.15.52:0/62963428 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67200679c0 0x7f6720069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.930+0000 7f6733fff640 1 --2- 172.21.15.52:0/62963428 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67200679c0 0x7f6720069e80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f672c0012f0 tx=0x7f672c001700 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.930+0000 7f6730ff9640 1 -- 172.21.15.52:0/62963428 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6728086180 con 0x7f6734070690 2024-05-04T12:18:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:28.930+0000 7f6730ff9640 1 -- 172.21.15.52:0/62963428 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f672801f020 con 0x7f6734070690 2024-05-04T12:18:29.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.138+0000 7f7655d45640 1 -- 172.21.15.52:0/2714785642 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f765006f590 con 0x7f765015afe0 2024-05-04T12:18:29.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.138+0000 7f76457fa640 1 -- 172.21.15.52:0/2714785642 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7650151290 con 0x7f765015afe0 2024-05-04T12:18:29.142 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-04T12:18:29.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.142+0000 7f7636ffd640 1 -- 172.21.15.52:0/2714785642 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f763c0678c0 msgr2=0x7f763c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.142+0000 7f7636ffd640 1 --2- 172.21.15.52:0/2714785642 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f763c0678c0 0x7f763c069d80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f764c01ab50 tx=0x7f764c008040 comp rx=0 tx=0).stop 2024-05-04T12:18:29.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.142+0000 7f7636ffd640 1 -- 172.21.15.52:0/2714785642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f765015afe0 msgr2=0x7f765014d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.142+0000 7f7636ffd640 1 --2- 172.21.15.52:0/2714785642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f765015afe0 0x7f765014d030 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f764800c760 tx=0x7f764800cc20 comp rx=0 tx=0).stop 2024-05-04T12:18:29.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.142+0000 7f7636ffd640 1 -- 172.21.15.52:0/2714785642 shutdown_connections 2024-05-04T12:18:29.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.142+0000 7f7636ffd640 1 --2- 172.21.15.52:0/2714785642 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f763c0678c0 0x7f763c069d80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.142+0000 7f7636ffd640 1 --2- 172.21.15.52:0/2714785642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f765015afe0 0x7f765014d030 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.142+0000 7f7636ffd640 1 --2- 172.21.15.52:0/2714785642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76500cdec0 0x7f765014f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.142+0000 7f7636ffd640 1 --2- 172.21.15.52:0/2714785642 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76500cdae0 0x7f765015d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.142+0000 7f7636ffd640 1 -- 172.21.15.52:0/2714785642 >> 172.21.15.52:0/2714785642 conn(0x7f76500b2010 msgr2=0x7f7650071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:29.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.142+0000 7f7636ffd640 1 -- 172.21.15.52:0/2714785642 shutdown_connections 2024-05-04T12:18:29.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.142+0000 7f7636ffd640 1 -- 172.21.15.52:0/2714785642 wait complete. 2024-05-04T12:18:29.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.150+0000 7f0f63f6d640 1 -- 172.21.15.52:0/3278330320 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0f5c075320 con 0x7f0f5c160190 2024-05-04T12:18:29.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.150+0000 7f0f527fc640 1 -- 172.21.15.52:0/3278330320 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0f5805a4e0 con 0x7f0f5c160190 2024-05-04T12:18:29.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.150+0000 7f14fb7db640 1 -- 172.21.15.52:0/1523808215 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f14d8005590 con 0x7f14e0003c10 2024-05-04T12:18:29.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.150+0000 7f14e9ffb640 1 -- 172.21.15.52:0/1523808215 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f14e40f3290 con 0x7f14e0003c10 2024-05-04T12:18:29.152 INFO:teuthology.orchestra.run.smithi052.stdout:55834574855 2024-05-04T12:18:29.152 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-04T12:18:29.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.154+0000 7f0f63f6d640 1 -- 172.21.15.52:0/3278330320 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0f400678c0 msgr2=0x7f0f40069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.154+0000 7f0f63f6d640 1 --2- 172.21.15.52:0/3278330320 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0f400678c0 0x7f0f40069d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c000f70 tx=0x7f0f4c007ea0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.154+0000 7f0f63f6d640 1 -- 172.21.15.52:0/3278330320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f5c160190 msgr2=0x7f0f5c155d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.154+0000 7f0f63f6d640 1 --2- 172.21.15.52:0/3278330320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f5c160190 0x7f0f5c155d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c1605a0 tx=0x7f0f5800c900 comp rx=0 tx=0).stop 2024-05-04T12:18:29.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.154+0000 7f0f63f6d640 1 -- 172.21.15.52:0/3278330320 shutdown_connections 2024-05-04T12:18:29.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.154+0000 7f0f63f6d640 1 --2- 172.21.15.52:0/3278330320 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0f400678c0 0x7f0f40069d80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.154+0000 7f0f63f6d640 1 --2- 172.21.15.52:0/3278330320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f5c160190 0x7f0f5c155d80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.154+0000 7f0f63f6d640 1 --2- 172.21.15.52:0/3278330320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f5c15fd00 0x7f0f5c204480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.154+0000 7f0f63f6d640 1 --2- 172.21.15.52:0/3278330320 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0f5c159ba0 0x7f0f5c1585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.154+0000 7f0f63f6d640 1 -- 172.21.15.52:0/3278330320 >> 172.21.15.52:0/3278330320 conn(0x7f0f5c067070 msgr2=0x7f0f5c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:29.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.154+0000 7f0f63f6d640 1 -- 172.21.15.52:0/3278330320 shutdown_connections 2024-05-04T12:18:29.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.154+0000 7f0f63f6d640 1 -- 172.21.15.52:0/3278330320 wait complete. 2024-05-04T12:18:29.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.158+0000 7f0a1b59e640 1 -- 172.21.15.52:0/682620463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0a1c06fc70 con 0x7f0a1c15e080 2024-05-04T12:18:29.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.158+0000 7f0a0affd640 1 -- 172.21.15.52:0/682620463 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0a0c055f80 con 0x7f0a1c15e080 2024-05-04T12:18:29.160 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-04T12:18:29.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.158+0000 7f14fb7db640 1 -- 172.21.15.52:0/1523808215 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f14e0069560 msgr2=0x7f14e006ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.158+0000 7f14fb7db640 1 --2- 172.21.15.52:0/1523808215 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f14e0069560 0x7f14e006ba20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f14f4159a80 tx=0x7f14f0008040 comp rx=0 tx=0).stop 2024-05-04T12:18:29.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.158+0000 7f14fb7db640 1 -- 172.21.15.52:0/1523808215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e0003c10 msgr2=0x7f14f40ad5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.158+0000 7f14fb7db640 1 --2- 172.21.15.52:0/1523808215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e0003c10 0x7f14f40ad5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14e401f100 tx=0x7f14e0005840 comp rx=0 tx=0).stop 2024-05-04T12:18:29.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.158+0000 7f14fb7db640 1 -- 172.21.15.52:0/1523808215 shutdown_connections 2024-05-04T12:18:29.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.158+0000 7f14fb7db640 1 --2- 172.21.15.52:0/1523808215 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f14e0069560 0x7f14e006ba20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.158+0000 7f14fb7db640 1 --2- 172.21.15.52:0/1523808215 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14f40ab900 0x7f14f40aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.158+0000 7f14fb7db640 1 --2- 172.21.15.52:0/1523808215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f40ab480 0x7f14f40a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.158+0000 7f14fb7db640 1 --2- 172.21.15.52:0/1523808215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e0003c10 0x7f14f40ad5c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.158+0000 7f14fb7db640 1 -- 172.21.15.52:0/1523808215 >> 172.21.15.52:0/1523808215 conn(0x7f14f4072d80 msgr2=0x7f14f4136090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:29.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.158+0000 7f14fb7db640 1 -- 172.21.15.52:0/1523808215 shutdown_connections 2024-05-04T12:18:29.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.158+0000 7f14fb7db640 1 -- 172.21.15.52:0/1523808215 wait complete. 2024-05-04T12:18:29.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.174+0000 7f0a08ff9640 1 -- 172.21.15.52:0/682620463 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f09fc0678c0 msgr2=0x7f09fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.174+0000 7f0a08ff9640 1 --2- 172.21.15.52:0/682620463 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f09fc0678c0 0x7f09fc069d80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0a140ae000 tx=0x7f0a140a7070 comp rx=0 tx=0).stop 2024-05-04T12:18:29.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.178+0000 7f0a08ff9640 1 -- 172.21.15.52:0/682620463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c15e080 msgr2=0x7f0a1c14a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.178+0000 7f0a08ff9640 1 --2- 172.21.15.52:0/682620463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c15e080 0x7f0a1c14a130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00b410 tx=0x7f0a0c009f90 comp rx=0 tx=0).stop 2024-05-04T12:18:29.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.178+0000 7f0a08ff9640 1 -- 172.21.15.52:0/682620463 shutdown_connections 2024-05-04T12:18:29.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.178+0000 7f0a08ff9640 1 --2- 172.21.15.52:0/682620463 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f09fc0678c0 0x7f09fc069d80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-05-04T12:18:29.180 DEBUG:teuthology.parallel:result is None 2024-05-04T12:18:29.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.178+0000 7f0a08ff9640 1 --2- 172.21.15.52:0/682620463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c15e080 0x7f0a1c14a130 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.178+0000 7f0a08ff9640 1 --2- 172.21.15.52:0/682620463 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a1c0cc880 0x7f0a1c149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.178+0000 7f0a08ff9640 1 --2- 172.21.15.52:0/682620463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a1c0b2de0 0x7f0a1c13ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.178+0000 7f0a08ff9640 1 -- 172.21.15.52:0/682620463 >> 172.21.15.52:0/682620463 conn(0x7f0a1c067070 msgr2=0x7f0a1c0ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:29.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.178+0000 7f0a08ff9640 1 -- 172.21.15.52:0/682620463 shutdown_connections 2024-05-04T12:18:29.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.178+0000 7f0a08ff9640 1 -- 172.21.15.52:0/682620463 wait complete. 2024-05-04T12:18:29.181 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-05-04T12:18:29.182 DEBUG:teuthology.parallel:result is None 2024-05-04T12:18:29.186 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-05-04T12:18:29.187 DEBUG:teuthology.parallel:result is None 2024-05-04T12:18:29.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.190+0000 7f3bbb66e640 1 -- 172.21.15.52:0/800672455 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3bb4075320 con 0x7f3bb406df80 2024-05-04T12:18:29.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.190+0000 7f3ba9ffb640 1 -- 172.21.15.52:0/800672455 <== mon.2 v2:172.21.15.111:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3ba4056e10 con 0x7f3bb406df80 2024-05-04T12:18:29.193 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-04T12:18:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.194+0000 7f3bbb66e640 1 -- 172.21.15.52:0/800672455 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f3ba0067a20 msgr2=0x7f3ba0069ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.194+0000 7f3bbb66e640 1 --2- 172.21.15.52:0/800672455 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f3ba0067a20 0x7f3ba0069ee0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3bac000f90 tx=0x7f3bac007d50 comp rx=0 tx=0).stop 2024-05-04T12:18:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.194+0000 7f3bbb66e640 1 -- 172.21.15.52:0/800672455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb406df80 msgr2=0x7f3bb414ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.194+0000 7f3bbb66e640 1 --2- 172.21.15.52:0/800672455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb406df80 0x7f3bb414ec20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400d830 tx=0x7f3ba400dcf0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.194+0000 7f3bbb66e640 1 -- 172.21.15.52:0/800672455 shutdown_connections 2024-05-04T12:18:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.194+0000 7f3bbb66e640 1 --2- 172.21.15.52:0/800672455 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f3ba0067a20 0x7f3ba0069ee0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.194+0000 7f3bbb66e640 1 --2- 172.21.15.52:0/800672455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bb415a210 0x7f3bb4168cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.194+0000 7f3bbb66e640 1 --2- 172.21.15.52:0/800672455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bb40cd1d0 0x7f3bb41687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.194+0000 7f3bbb66e640 1 --2- 172.21.15.52:0/800672455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb406df80 0x7f3bb414ec20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.194+0000 7f3bbb66e640 1 -- 172.21.15.52:0/800672455 >> 172.21.15.52:0/800672455 conn(0x7f3bb4067070 msgr2=0x7f3bb415a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.194+0000 7f3bbb66e640 1 -- 172.21.15.52:0/800672455 shutdown_connections 2024-05-04T12:18:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.194+0000 7f3bbb66e640 1 -- 172.21.15.52:0/800672455 wait complete. 2024-05-04T12:18:29.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-05-04T12:18:29.203 DEBUG:teuthology.parallel:result is None 2024-05-04T12:18:29.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-05-04T12:18:29.211 DEBUG:teuthology.parallel:result is None 2024-05-04T12:18:29.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.214+0000 7f5f62bf7640 1 -- 172.21.15.52:0/3482405466 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f5f40005590 con 0x7f5f5c14bd60 2024-05-04T12:18:29.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f5f527fc640 1 -- 172.21.15.52:0/3482405466 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5f5405d460 con 0x7f5f5c14bd60 2024-05-04T12:18:29.219 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-04T12:18:29.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f25053d7640 1 -- 172.21.15.52:0/3645570885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f24e0005590 con 0x7f25000cefa0 2024-05-04T12:18:29.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f5f62bf7640 1 -- 172.21.15.52:0/3482405466 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5f48067a10 msgr2=0x7f5f48069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f5f62bf7640 1 --2- 172.21.15.52:0/3482405466 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5f48067a10 0x7f5f48069ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5f580ad000 tx=0x7f5f580a6000 comp rx=0 tx=0).stop 2024-05-04T12:18:29.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f5f62bf7640 1 -- 172.21.15.52:0/3482405466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f5c14bd60 msgr2=0x7f5f5c163840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f5f62bf7640 1 --2- 172.21.15.52:0/3482405466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f5c14bd60 0x7f5f5c163840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400e480 tx=0x7f5f5400e940 comp rx=0 tx=0).stop 2024-05-04T12:18:29.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f5f62bf7640 1 -- 172.21.15.52:0/3482405466 shutdown_connections 2024-05-04T12:18:29.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f5f62bf7640 1 --2- 172.21.15.52:0/3482405466 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5f48067a10 0x7f5f48069ed0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f5f62bf7640 1 --2- 172.21.15.52:0/3482405466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f5c14bd60 0x7f5f5c163840 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f5f62bf7640 1 --2- 172.21.15.52:0/3482405466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f5c0cd630 0x7f5f5c163300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f5f62bf7640 1 --2- 172.21.15.52:0/3482405466 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f5c0b2de0 0x7f5f5c071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f5f62bf7640 1 -- 172.21.15.52:0/3482405466 >> 172.21.15.52:0/3482405466 conn(0x7f5f5c067070 msgr2=0x7f5f5c0b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:29.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f24e77fe640 1 -- 172.21.15.52:0/3645570885 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f24f40565e0 con 0x7f25000cefa0 2024-05-04T12:18:29.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f5f62bf7640 1 -- 172.21.15.52:0/3482405466 shutdown_connections 2024-05-04T12:18:29.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.218+0000 7f5f62bf7640 1 -- 172.21.15.52:0/3482405466 wait complete. 2024-05-04T12:18:29.223 INFO:teuthology.orchestra.run.smithi052.stdout:55834574855 2024-05-04T12:18:29.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.222+0000 7f25053d7640 1 -- 172.21.15.52:0/3645570885 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f24ec0678c0 msgr2=0x7f24ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.222+0000 7f25053d7640 1 --2- 172.21.15.52:0/3645570885 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f24ec0678c0 0x7f24ec069d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24f001c310 tx=0x7f24f00a6000 comp rx=0 tx=0).stop 2024-05-04T12:18:29.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.222+0000 7f25053d7640 1 -- 172.21.15.52:0/3645570885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25000cefa0 msgr2=0x7f25002046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.222+0000 7f25053d7640 1 --2- 172.21.15.52:0/3645570885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25000cefa0 0x7f25002046c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f24f4011960 tx=0x7f24f4011e20 comp rx=0 tx=0).stop 2024-05-04T12:18:29.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.222+0000 7f25053d7640 1 -- 172.21.15.52:0/3645570885 shutdown_connections 2024-05-04T12:18:29.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.222+0000 7f25053d7640 1 --2- 172.21.15.52:0/3645570885 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f24ec0678c0 0x7f24ec069d80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.222+0000 7f25053d7640 1 --2- 172.21.15.52:0/3645570885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2500165090 0x7f2500204c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.222+0000 7f25053d7640 1 --2- 172.21.15.52:0/3645570885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25000cefa0 0x7f25002046c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.222+0000 7f25053d7640 1 --2- 172.21.15.52:0/3645570885 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2500164cb0 0x7f25000cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.222+0000 7f25053d7640 1 -- 172.21.15.52:0/3645570885 >> 172.21.15.52:0/3645570885 conn(0x7f25000b14a0 msgr2=0x7f25000b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:29.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.222+0000 7f25053d7640 1 -- 172.21.15.52:0/3645570885 shutdown_connections 2024-05-04T12:18:29.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.222+0000 7f25053d7640 1 -- 172.21.15.52:0/3645570885 wait complete. 2024-05-04T12:18:29.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-05-04T12:18:29.239 DEBUG:teuthology.parallel:result is None 2024-05-04T12:18:29.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-05-04T12:18:29.241 DEBUG:teuthology.parallel:result is None 2024-05-04T12:18:29.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.242+0000 7f673a83b640 1 -- 172.21.15.52:0/62963428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f67341411d0 con 0x7f6734070690 2024-05-04T12:18:29.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.242+0000 7f6730ff9640 1 -- 172.21.15.52:0/62963428 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6728056650 con 0x7f6734070690 2024-05-04T12:18:29.246 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-04T12:18:29.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.246+0000 7f673a83b640 1 -- 172.21.15.52:0/62963428 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67200679c0 msgr2=0x7f6720069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.246+0000 7f673a83b640 1 --2- 172.21.15.52:0/62963428 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67200679c0 0x7f6720069e80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f672c0012f0 tx=0x7f672c001700 comp rx=0 tx=0).stop 2024-05-04T12:18:29.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.246+0000 7f673a83b640 1 -- 172.21.15.52:0/62963428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6734070690 msgr2=0x7f67340c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.246+0000 7f673a83b640 1 --2- 172.21.15.52:0/62963428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6734070690 0x7f67340c9a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f672800b3f0 tx=0x7f672800b8b0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.246+0000 7f673a83b640 1 -- 172.21.15.52:0/62963428 shutdown_connections 2024-05-04T12:18:29.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.246+0000 7f673a83b640 1 --2- 172.21.15.52:0/62963428 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67200679c0 0x7f6720069e80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.246+0000 7f673a83b640 1 --2- 172.21.15.52:0/62963428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f673414bbf0 0x7f67340c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.246+0000 7f673a83b640 1 --2- 172.21.15.52:0/62963428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6734070690 0x7f67340c9a00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.246+0000 7f673a83b640 1 --2- 172.21.15.52:0/62963428 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f673406f4e0 0x7f67340cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.246+0000 7f673a83b640 1 -- 172.21.15.52:0/62963428 >> 172.21.15.52:0/62963428 conn(0x7f6734072d80 msgr2=0x7f6734163000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:29.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.246+0000 7f673a83b640 1 -- 172.21.15.52:0/62963428 shutdown_connections 2024-05-04T12:18:29.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.246+0000 7f673a83b640 1 -- 172.21.15.52:0/62963428 wait complete. 2024-05-04T12:18:29.263 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-05-04T12:18:29.263 DEBUG:teuthology.parallel:result is None 2024-05-04T12:18:29.263 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-04T12:18:29.263 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-04T12:18:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.410+0000 7fe0cea34640 1 Processor -- start 2024-05-04T12:18:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.410+0000 7fe0cea34640 1 -- start start 2024-05-04T12:18:29.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0cea34640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe0c815bf50 0x7fe0c815c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:29.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0cea34640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe0c80aa0c0 0x7fe0c80ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:29.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0cea34640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c80aa550 0x7fe0c80cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:29.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0cea34640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c806d1b0 con 0x7fe0c80aa0c0 2024-05-04T12:18:29.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0cea34640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c806d490 con 0x7fe0c815bf50 2024-05-04T12:18:29.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0cea34640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c806d770 con 0x7fe0c80aa550 2024-05-04T12:18:29.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0c77fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe0c80aa0c0 0x7fe0c80ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:29.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0c77fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe0c80aa0c0 0x7fe0c80ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53606/0 (socket says 172.21.15.52:53606) 2024-05-04T12:18:29.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0c77fe640 1 -- 172.21.15.52:0/2373708991 learned_addr learned my addr 172.21.15.52:0/2373708991 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:29.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0ccfd1640 1 --2- 172.21.15.52:0/2373708991 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c80aa550 0x7fe0c80cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:29.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0c7fff640 1 --2- 172.21.15.52:0/2373708991 >> v2:172.21.15.111:3300/0 conn(0x7fe0c815bf50 0x7fe0c815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:29.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0c77fe640 1 -- 172.21.15.52:0/2373708991 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c80aa550 msgr2=0x7fe0c80cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:29.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0c77fe640 1 --2- 172.21.15.52:0/2373708991 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c80aa550 0x7fe0c80cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0c77fe640 1 -- 172.21.15.52:0/2373708991 >> v2:172.21.15.111:3300/0 conn(0x7fe0c815bf50 msgr2=0x7fe0c815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:29.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0c77fe640 1 --2- 172.21.15.52:0/2373708991 >> v2:172.21.15.111:3300/0 conn(0x7fe0c815bf50 0x7fe0c815c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.414+0000 7fe0c77fe640 1 -- 172.21.15.52:0/2373708991 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c806da50 con 0x7fe0c80aa0c0 2024-05-04T12:18:29.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0c77fe640 1 --2- 172.21.15.52:0/2373708991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0c80aa0c0 0x7fe0c80ccb50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8069260 tx=0x7fe0b001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f5deee34283eb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:29.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0c6ffd640 1 -- 172.21.15.52:0/2373708991 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00a5860 con 0x7fe0c80aa0c0 2024-05-04T12:18:29.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0c6ffd640 1 -- 172.21.15.52:0/2373708991 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b00a5a00 con 0x7fe0c80aa0c0 2024-05-04T12:18:29.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0c6ffd640 1 -- 172.21.15.52:0/2373708991 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00a5dd0 con 0x7fe0c80aa0c0 2024-05-04T12:18:29.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 -- 172.21.15.52:0/2373708991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0c80aa0c0 msgr2=0x7fe0c80ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 --2- 172.21.15.52:0/2373708991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0c80aa0c0 0x7fe0c80ccb50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8069260 tx=0x7fe0b001c9c0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 -- 172.21.15.52:0/2373708991 shutdown_connections 2024-05-04T12:18:29.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 --2- 172.21.15.52:0/2373708991 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c80aa550 0x7fe0c80cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 --2- 172.21.15.52:0/2373708991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0c80aa0c0 0x7fe0c80ccb50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 --2- 172.21.15.52:0/2373708991 >> v2:172.21.15.111:3300/0 conn(0x7fe0c815bf50 0x7fe0c815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 -- 172.21.15.52:0/2373708991 >> 172.21.15.52:0/2373708991 conn(0x7fe0c80b14a0 msgr2=0x7fe0c80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:29.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 -- 172.21.15.52:0/2373708991 shutdown_connections 2024-05-04T12:18:29.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 -- 172.21.15.52:0/2373708991 wait complete. 2024-05-04T12:18:29.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 Processor -- start 2024-05-04T12:18:29.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 -- start start 2024-05-04T12:18:29.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c80aa0c0 0x7fe0c81481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:29.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0c80aa550 0x7fe0c8151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:29.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0c815bf50 0x7fe0c80ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:29.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c806ebe0 con 0x7fe0c815bf50 2024-05-04T12:18:29.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c806ea60 con 0x7fe0c80aa550 2024-05-04T12:18:29.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0cea34640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c806ed60 con 0x7fe0c80aa0c0 2024-05-04T12:18:29.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0ccfd1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0c815bf50 0x7fe0c80ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:29.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0ccfd1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0c815bf50 0x7fe0c80ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53614/0 (socket says 172.21.15.52:53614) 2024-05-04T12:18:29.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.418+0000 7fe0ccfd1640 1 -- 172.21.15.52:0/1401372615 learned_addr learned my addr 172.21.15.52:0/1401372615 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:29.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0c77fe640 1 --2- 172.21.15.52:0/1401372615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0c80aa550 0x7fe0c8151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:29.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0ccfd1640 1 -- 172.21.15.52:0/1401372615 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c80aa0c0 msgr2=0x7fe0c81481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0ccfd1640 1 --2- 172.21.15.52:0/1401372615 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c80aa0c0 0x7fe0c81481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0ccfd1640 1 -- 172.21.15.52:0/1401372615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0c80aa550 msgr2=0x7fe0c8151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0ccfd1640 1 --2- 172.21.15.52:0/1401372615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0c80aa550 0x7fe0c8151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0ccfd1640 1 -- 172.21.15.52:0/1401372615 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c8151bf0 con 0x7fe0c815bf50 2024-05-04T12:18:29.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0c77fe640 1 --2- 172.21.15.52:0/1401372615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0c80aa550 0x7fe0c8151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:29.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0ccfd1640 1 --2- 172.21.15.52:0/1401372615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0c815bf50 0x7fe0c80ce050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800d830 tx=0x7fe0b800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:29.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0c4ff9640 1 -- 172.21.15.52:0/1401372615 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8018020 con 0x7fe0c815bf50 2024-05-04T12:18:29.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0c4ff9640 1 -- 172.21.15.52:0/1401372615 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b80096c0 con 0x7fe0c815bf50 2024-05-04T12:18:29.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0cea34640 1 -- 172.21.15.52:0/1401372615 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c8151ed0 con 0x7fe0c815bf50 2024-05-04T12:18:29.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0c4ff9640 1 -- 172.21.15.52:0/1401372615 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8009ab0 con 0x7fe0c815bf50 2024-05-04T12:18:29.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0cea34640 1 -- 172.21.15.52:0/1401372615 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c8152490 con 0x7fe0c815bf50 2024-05-04T12:18:29.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0cea34640 1 -- 172.21.15.52:0/1401372615 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c8062e50 con 0x7fe0c815bf50 2024-05-04T12:18:29.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0c4ff9640 1 -- 172.21.15.52:0/1401372615 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe0b801b020 con 0x7fe0c815bf50 2024-05-04T12:18:29.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0c4ff9640 1 --2- 172.21.15.52:0/1401372615 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe0a80678f0 0x7fe0a8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:29.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0c4ff9640 1 -- 172.21.15.52:0/1401372615 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe0b8086210 con 0x7fe0c815bf50 2024-05-04T12:18:29.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0c7fff640 1 --2- 172.21.15.52:0/1401372615 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe0a80678f0 0x7fe0a8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:29.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.422+0000 7fe0c7fff640 1 --2- 172.21.15.52:0/1401372615 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe0a80678f0 0x7fe0a8069db0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000b850 tx=0x7fe0c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:29.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.426+0000 7fe0c4ff9640 1 -- 172.21.15.52:0/1401372615 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe0b8022020 con 0x7fe0c815bf50 2024-05-04T12:18:29.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.578+0000 7fe0cea34640 1 -- 172.21.15.52:0/1401372615 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0c8075320 con 0x7fe0a80678f0 2024-05-04T12:18:29.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.594+0000 7fe0c4ff9640 1 -- 172.21.15.52:0/1401372615 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251872 (secure 0 0 0) 0x7fe0c8075320 con 0x7fe0a80678f0 2024-05-04T12:18:29.600 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-04T12:18:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.598+0000 7fe0cea34640 1 -- 172.21.15.52:0/1401372615 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe0a80678f0 msgr2=0x7fe0a8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.598+0000 7fe0cea34640 1 --2- 172.21.15.52:0/1401372615 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe0a80678f0 0x7fe0a8069db0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000b850 tx=0x7fe0c0008040 comp rx=0 tx=0).stop 2024-05-04T12:18:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.598+0000 7fe0cea34640 1 -- 172.21.15.52:0/1401372615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0c815bf50 msgr2=0x7fe0c80ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.598+0000 7fe0cea34640 1 --2- 172.21.15.52:0/1401372615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0c815bf50 0x7fe0c80ce050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800d830 tx=0x7fe0b800dcf0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.598+0000 7fe0cea34640 1 -- 172.21.15.52:0/1401372615 shutdown_connections 2024-05-04T12:18:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.598+0000 7fe0cea34640 1 --2- 172.21.15.52:0/1401372615 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe0a80678f0 0x7fe0a8069db0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.598+0000 7fe0cea34640 1 --2- 172.21.15.52:0/1401372615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0c815bf50 0x7fe0c80ce050 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.598+0000 7fe0cea34640 1 --2- 172.21.15.52:0/1401372615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0c80aa550 0x7fe0c8151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.598+0000 7fe0cea34640 1 --2- 172.21.15.52:0/1401372615 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c80aa0c0 0x7fe0c81481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.598+0000 7fe0cea34640 1 -- 172.21.15.52:0/1401372615 >> 172.21.15.52:0/1401372615 conn(0x7fe0c80b14a0 msgr2=0x7fe0c80cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.598+0000 7fe0cea34640 1 -- 172.21.15.52:0/1401372615 shutdown_connections 2024-05-04T12:18:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.598+0000 7fe0cea34640 1 -- 172.21.15.52:0/1401372615 wait complete. 2024-05-04T12:18:29.617 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-04T12:18:28.741660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235720,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739000,"statfs":{"total":773094113280,"available":772852736000,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.932093"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298061+0000","last_change":"2024-05-04T12:18:22.859306+0000","last_active":"2024-05-04T12:18:24.298061+0000","last_peered":"2024-05-04T12:18:24.298061+0000","last_clean":"2024-05-04T12:18:24.298061+0000","last_became_active":"2024-05-04T12:18:22.859107+0000","last_became_peered":"2024-05-04T12:18:22.859107+0000","last_unstale":"2024-05-04T12:18:24.298061+0000","last_undegraded":"2024-05-04T12:18:24.298061+0000","last_fullsized":"2024-05-04T12:18:24.298061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:45:17.602528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.832155+0000","last_change":"2024-05-04T12:18:22.848885+0000","last_active":"2024-05-04T12:18:23.832155+0000","last_peered":"2024-05-04T12:18:23.832155+0000","last_clean":"2024-05-04T12:18:23.832155+0000","last_became_active":"2024-05-04T12:18:22.848320+0000","last_became_peered":"2024-05-04T12:18:22.848320+0000","last_unstale":"2024-05-04T12:18:23.832155+0000","last_undegraded":"2024-05-04T12:18:23.832155+0000","last_fullsized":"2024-05-04T12:18:23.832155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247882+0000","last_change":"2024-05-04T12:18:22.861471+0000","last_active":"2024-05-04T12:18:24.247882+0000","last_peered":"2024-05-04T12:18:24.247882+0000","last_clean":"2024-05-04T12:18:24.247882+0000","last_became_active":"2024-05-04T12:18:22.860827+0000","last_became_peered":"2024-05-04T12:18:22.860827+0000","last_unstale":"2024-05-04T12:18:24.247882+0000","last_undegraded":"2024-05-04T12:18:24.247882+0000","last_fullsized":"2024-05-04T12:18:24.247882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:10:47.797595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836758+0000","last_change":"2024-05-04T12:18:22.878704+0000","last_active":"2024-05-04T12:18:23.836758+0000","last_peered":"2024-05-04T12:18:23.836758+0000","last_clean":"2024-05-04T12:18:23.836758+0000","last_became_active":"2024-05-04T12:18:22.878423+0000","last_became_peered":"2024-05-04T12:18:22.878423+0000","last_unstale":"2024-05-04T12:18:23.836758+0000","last_undegraded":"2024-05-04T12:18:23.836758+0000","last_fullsized":"2024-05-04T12:18:23.836758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:28:22.639254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297059+0000","last_change":"2024-05-04T12:18:22.854847+0000","last_active":"2024-05-04T12:18:24.297059+0000","last_peered":"2024-05-04T12:18:24.297059+0000","last_clean":"2024-05-04T12:18:24.297059+0000","last_became_active":"2024-05-04T12:18:22.854509+0000","last_became_peered":"2024-05-04T12:18:22.854509+0000","last_unstale":"2024-05-04T12:18:24.297059+0000","last_undegraded":"2024-05-04T12:18:24.297059+0000","last_fullsized":"2024-05-04T12:18:24.297059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:05:46.918761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835635+0000","last_change":"2024-05-04T12:18:22.856252+0000","last_active":"2024-05-04T12:18:23.835635+0000","last_peered":"2024-05-04T12:18:23.835635+0000","last_clean":"2024-05-04T12:18:23.835635+0000","last_became_active":"2024-05-04T12:18:22.855950+0000","last_became_peered":"2024-05-04T12:18:22.855950+0000","last_unstale":"2024-05-04T12:18:23.835635+0000","last_undegraded":"2024-05-04T12:18:23.835635+0000","last_fullsized":"2024-05-04T12:18:23.835635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:10:47.797595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297649+0000","last_change":"2024-05-04T12:18:22.851916+0000","last_active":"2024-05-04T12:18:24.297649+0000","last_peered":"2024-05-04T12:18:24.297649+0000","last_clean":"2024-05-04T12:18:24.297649+0000","last_became_active":"2024-05-04T12:18:22.851291+0000","last_became_peered":"2024-05-04T12:18:22.851291+0000","last_unstale":"2024-05-04T12:18:24.297649+0000","last_undegraded":"2024-05-04T12:18:24.297649+0000","last_fullsized":"2024-05-04T12:18:24.297649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:02:37.747970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778425+0000","last_change":"2024-05-04T12:18:22.879929+0000","last_active":"2024-05-04T12:18:24.778425+0000","last_peered":"2024-05-04T12:18:24.778425+0000","last_clean":"2024-05-04T12:18:24.778425+0000","last_became_active":"2024-05-04T12:18:22.879364+0000","last_became_peered":"2024-05-04T12:18:22.879364+0000","last_unstale":"2024-05-04T12:18:24.778425+0000","last_undegraded":"2024-05-04T12:18:24.778425+0000","last_fullsized":"2024-05-04T12:18:24.778425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:51.416179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298129+0000","last_change":"2024-05-04T12:18:22.859239+0000","last_active":"2024-05-04T12:18:24.298129+0000","last_peered":"2024-05-04T12:18:24.298129+0000","last_clean":"2024-05-04T12:18:24.298129+0000","last_became_active":"2024-05-04T12:18:22.858831+0000","last_became_peered":"2024-05-04T12:18:22.858831+0000","last_unstale":"2024-05-04T12:18:24.298129+0000","last_undegraded":"2024-05-04T12:18:24.298129+0000","last_fullsized":"2024-05-04T12:18:24.298129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:09:40.457188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777413+0000","last_change":"2024-05-04T12:18:22.853885+0000","last_active":"2024-05-04T12:18:24.777413+0000","last_peered":"2024-05-04T12:18:24.777413+0000","last_clean":"2024-05-04T12:18:24.777413+0000","last_became_active":"2024-05-04T12:18:22.853589+0000","last_became_peered":"2024-05-04T12:18:22.853589+0000","last_unstale":"2024-05-04T12:18:24.777413+0000","last_undegraded":"2024-05-04T12:18:24.777413+0000","last_fullsized":"2024-05-04T12:18:24.777413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777354+0000","last_change":"2024-05-04T12:18:22.856994+0000","last_active":"2024-05-04T12:18:24.777354+0000","last_peered":"2024-05-04T12:18:24.777354+0000","last_clean":"2024-05-04T12:18:24.777354+0000","last_became_active":"2024-05-04T12:18:22.856651+0000","last_became_peered":"2024-05-04T12:18:22.856651+0000","last_unstale":"2024-05-04T12:18:24.777354+0000","last_undegraded":"2024-05-04T12:18:24.777354+0000","last_fullsized":"2024-05-04T12:18:24.777354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836324+0000","last_change":"2024-05-04T12:18:22.864231+0000","last_active":"2024-05-04T12:18:23.836324+0000","last_peered":"2024-05-04T12:18:23.836324+0000","last_clean":"2024-05-04T12:18:23.836324+0000","last_became_active":"2024-05-04T12:18:22.861558+0000","last_became_peered":"2024-05-04T12:18:22.861558+0000","last_unstale":"2024-05-04T12:18:23.836324+0000","last_undegraded":"2024-05-04T12:18:23.836324+0000","last_fullsized":"2024-05-04T12:18:23.836324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:37.729658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779374+0000","last_change":"2024-05-04T12:18:22.868189+0000","last_active":"2024-05-04T12:18:24.779374+0000","last_peered":"2024-05-04T12:18:24.779374+0000","last_clean":"2024-05-04T12:18:24.779374+0000","last_became_active":"2024-05-04T12:18:22.860456+0000","last_became_peered":"2024-05-04T12:18:22.860456+0000","last_unstale":"2024-05-04T12:18:24.779374+0000","last_undegraded":"2024-05-04T12:18:24.779374+0000","last_fullsized":"2024-05-04T12:18:24.779374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:07:22.005194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.246875+0000","last_change":"2024-05-04T12:18:22.853777+0000","last_active":"2024-05-04T12:18:24.246875+0000","last_peered":"2024-05-04T12:18:24.246875+0000","last_clean":"2024-05-04T12:18:24.246875+0000","last_became_active":"2024-05-04T12:18:22.852626+0000","last_became_peered":"2024-05-04T12:18:22.852626+0000","last_unstale":"2024-05-04T12:18:24.246875+0000","last_undegraded":"2024-05-04T12:18:24.246875+0000","last_fullsized":"2024-05-04T12:18:24.246875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.833385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831334+0000","last_change":"2024-05-04T12:18:22.844392+0000","last_active":"2024-05-04T12:18:23.831334+0000","last_peered":"2024-05-04T12:18:23.831334+0000","last_clean":"2024-05-04T12:18:23.831334+0000","last_became_active":"2024-05-04T12:18:22.843539+0000","last_became_peered":"2024-05-04T12:18:22.843539+0000","last_unstale":"2024-05-04T12:18:23.831334+0000","last_undegraded":"2024-05-04T12:18:23.831334+0000","last_fullsized":"2024-05-04T12:18:23.831334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:18:08.570317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.834671+0000","last_change":"2024-05-04T12:18:22.849846+0000","last_active":"2024-05-04T12:18:23.834671+0000","last_peered":"2024-05-04T12:18:23.834671+0000","last_clean":"2024-05-04T12:18:23.834671+0000","last_became_active":"2024-05-04T12:18:22.849552+0000","last_became_peered":"2024-05-04T12:18:22.849552+0000","last_unstale":"2024-05-04T12:18:23.834671+0000","last_undegraded":"2024-05-04T12:18:23.834671+0000","last_fullsized":"2024-05-04T12:18:23.834671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.837479+0000","last_change":"2024-05-04T12:18:22.851494+0000","last_active":"2024-05-04T12:18:23.837479+0000","last_peered":"2024-05-04T12:18:23.837479+0000","last_clean":"2024-05-04T12:18:23.837479+0000","last_became_active":"2024-05-04T12:18:22.851177+0000","last_became_peered":"2024-05-04T12:18:22.851177+0000","last_unstale":"2024-05-04T12:18:23.837479+0000","last_undegraded":"2024-05-04T12:18:23.837479+0000","last_fullsized":"2024-05-04T12:18:23.837479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:52:05.609251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778115+0000","last_change":"2024-05-04T12:18:22.861585+0000","last_active":"2024-05-04T12:18:24.778115+0000","last_peered":"2024-05-04T12:18:24.778115+0000","last_clean":"2024-05-04T12:18:24.778115+0000","last_became_active":"2024-05-04T12:18:22.861304+0000","last_became_peered":"2024-05-04T12:18:22.861304+0000","last_unstale":"2024-05-04T12:18:24.778115+0000","last_undegraded":"2024-05-04T12:18:24.778115+0000","last_fullsized":"2024-05-04T12:18:24.778115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:40.494337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297947+0000","last_change":"2024-05-04T12:18:22.855243+0000","last_active":"2024-05-04T12:18:24.297947+0000","last_peered":"2024-05-04T12:18:24.297947+0000","last_clean":"2024-05-04T12:18:24.297947+0000","last_became_active":"2024-05-04T12:18:22.854938+0000","last_became_peered":"2024-05-04T12:18:22.854938+0000","last_unstale":"2024-05-04T12:18:24.297947+0000","last_undegraded":"2024-05-04T12:18:24.297947+0000","last_fullsized":"2024-05-04T12:18:24.297947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:52:05.609251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777201+0000","last_change":"2024-05-04T12:18:22.870381+0000","last_active":"2024-05-04T12:18:24.777201+0000","last_peered":"2024-05-04T12:18:24.777201+0000","last_clean":"2024-05-04T12:18:24.777201+0000","last_became_active":"2024-05-04T12:18:22.870076+0000","last_became_peered":"2024-05-04T12:18:22.870076+0000","last_unstale":"2024-05-04T12:18:24.777201+0000","last_undegraded":"2024-05-04T12:18:24.777201+0000","last_fullsized":"2024-05-04T12:18:24.777201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.636826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247261+0000","last_change":"2024-05-04T12:18:22.853194+0000","last_active":"2024-05-04T12:18:24.247261+0000","last_peered":"2024-05-04T12:18:24.247261+0000","last_clean":"2024-05-04T12:18:24.247261+0000","last_became_active":"2024-05-04T12:18:22.852625+0000","last_became_peered":"2024-05-04T12:18:22.852625+0000","last_unstale":"2024-05-04T12:18:24.247261+0000","last_undegraded":"2024-05-04T12:18:24.247261+0000","last_fullsized":"2024-05-04T12:18:24.247261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:51.416179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836035+0000","last_change":"2024-05-04T12:18:22.844120+0000","last_active":"2024-05-04T12:18:23.836035+0000","last_peered":"2024-05-04T12:18:23.836035+0000","last_clean":"2024-05-04T12:18:23.836035+0000","last_became_active":"2024-05-04T12:18:22.841231+0000","last_became_peered":"2024-05-04T12:18:22.841231+0000","last_unstale":"2024-05-04T12:18:23.836035+0000","last_undegraded":"2024-05-04T12:18:23.836035+0000","last_fullsized":"2024-05-04T12:18:23.836035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:07:22.005194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778499+0000","last_change":"2024-05-04T12:18:22.877765+0000","last_active":"2024-05-04T12:18:24.778499+0000","last_peered":"2024-05-04T12:18:24.778499+0000","last_clean":"2024-05-04T12:18:24.778499+0000","last_became_active":"2024-05-04T12:18:22.877351+0000","last_became_peered":"2024-05-04T12:18:22.877351+0000","last_unstale":"2024-05-04T12:18:24.778499+0000","last_undegraded":"2024-05-04T12:18:24.778499+0000","last_fullsized":"2024-05-04T12:18:24.778499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:18:08.570317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.834935+0000","last_change":"2024-05-04T12:18:22.878164+0000","last_active":"2024-05-04T12:18:23.834935+0000","last_peered":"2024-05-04T12:18:23.834935+0000","last_clean":"2024-05-04T12:18:23.834935+0000","last_became_active":"2024-05-04T12:18:22.877881+0000","last_became_peered":"2024-05-04T12:18:22.877881+0000","last_unstale":"2024-05-04T12:18:23.834935+0000","last_undegraded":"2024-05-04T12:18:23.834935+0000","last_fullsized":"2024-05-04T12:18:23.834935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778748+0000","last_change":"2024-05-04T12:18:22.880454+0000","last_active":"2024-05-04T12:18:24.778748+0000","last_peered":"2024-05-04T12:18:24.778748+0000","last_clean":"2024-05-04T12:18:24.778748+0000","last_became_active":"2024-05-04T12:18:22.877223+0000","last_became_peered":"2024-05-04T12:18:22.877223+0000","last_unstale":"2024-05-04T12:18:24.778748+0000","last_undegraded":"2024-05-04T12:18:24.778748+0000","last_fullsized":"2024-05-04T12:18:24.778748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.636826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777595+0000","last_change":"2024-05-04T12:18:22.862461+0000","last_active":"2024-05-04T12:18:24.777595+0000","last_peered":"2024-05-04T12:18:24.777595+0000","last_clean":"2024-05-04T12:18:24.777595+0000","last_became_active":"2024-05-04T12:18:22.862122+0000","last_became_peered":"2024-05-04T12:18:22.862122+0000","last_unstale":"2024-05-04T12:18:24.777595+0000","last_undegraded":"2024-05-04T12:18:24.777595+0000","last_fullsized":"2024-05-04T12:18:24.777595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:10:47.797595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831950+0000","last_change":"2024-05-04T12:18:22.870835+0000","last_active":"2024-05-04T12:18:23.831950+0000","last_peered":"2024-05-04T12:18:23.831950+0000","last_clean":"2024-05-04T12:18:23.831950+0000","last_became_active":"2024-05-04T12:18:22.870544+0000","last_became_peered":"2024-05-04T12:18:22.870544+0000","last_unstale":"2024-05-04T12:18:23.831950+0000","last_undegraded":"2024-05-04T12:18:23.831950+0000","last_fullsized":"2024-05-04T12:18:23.831950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779872+0000","last_change":"2024-05-04T12:18:22.869151+0000","last_active":"2024-05-04T12:18:24.779872+0000","last_peered":"2024-05-04T12:18:24.779872+0000","last_clean":"2024-05-04T12:18:24.779872+0000","last_became_active":"2024-05-04T12:18:22.868856+0000","last_became_peered":"2024-05-04T12:18:22.868856+0000","last_unstale":"2024-05-04T12:18:24.779872+0000","last_undegraded":"2024-05-04T12:18:24.779872+0000","last_fullsized":"2024-05-04T12:18:24.779872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:28:22.639254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247351+0000","last_change":"2024-05-04T12:18:22.873982+0000","last_active":"2024-05-04T12:18:24.247351+0000","last_peered":"2024-05-04T12:18:24.247351+0000","last_clean":"2024-05-04T12:18:24.247351+0000","last_became_active":"2024-05-04T12:18:22.871687+0000","last_became_peered":"2024-05-04T12:18:22.871687+0000","last_unstale":"2024-05-04T12:18:24.247351+0000","last_undegraded":"2024-05-04T12:18:24.247351+0000","last_fullsized":"2024-05-04T12:18:24.247351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:47:19.879004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836157+0000","last_change":"2024-05-04T12:18:22.878584+0000","last_active":"2024-05-04T12:18:23.836157+0000","last_peered":"2024-05-04T12:18:23.836157+0000","last_clean":"2024-05-04T12:18:23.836157+0000","last_became_active":"2024-05-04T12:18:22.878093+0000","last_became_peered":"2024-05-04T12:18:22.878093+0000","last_unstale":"2024-05-04T12:18:23.836157+0000","last_undegraded":"2024-05-04T12:18:23.836157+0000","last_fullsized":"2024-05-04T12:18:23.836157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:40:13.207623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777628+0000","last_change":"2024-05-04T12:18:22.851028+0000","last_active":"2024-05-04T12:18:24.777628+0000","last_peered":"2024-05-04T12:18:24.777628+0000","last_clean":"2024-05-04T12:18:24.777628+0000","last_became_active":"2024-05-04T12:18:22.850280+0000","last_became_peered":"2024-05-04T12:18:22.850280+0000","last_unstale":"2024-05-04T12:18:24.777628+0000","last_undegraded":"2024-05-04T12:18:24.777628+0000","last_fullsized":"2024-05-04T12:18:24.777628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:37.729658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.776877+0000","last_change":"2024-05-04T12:18:22.870893+0000","last_active":"2024-05-04T12:18:24.776877+0000","last_peered":"2024-05-04T12:18:24.776877+0000","last_clean":"2024-05-04T12:18:24.776877+0000","last_became_active":"2024-05-04T12:18:22.870517+0000","last_became_peered":"2024-05-04T12:18:22.870517+0000","last_unstale":"2024-05-04T12:18:24.776877+0000","last_undegraded":"2024-05-04T12:18:24.776877+0000","last_fullsized":"2024-05-04T12:18:24.776877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:51.416179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835952+0000","last_change":"2024-05-04T12:18:22.840511+0000","last_active":"2024-05-04T12:18:23.835952+0000","last_peered":"2024-05-04T12:18:23.835952+0000","last_clean":"2024-05-04T12:18:23.835952+0000","last_became_active":"2024-05-04T12:18:22.839811+0000","last_became_peered":"2024-05-04T12:18:22.839811+0000","last_unstale":"2024-05-04T12:18:23.835952+0000","last_undegraded":"2024-05-04T12:18:23.835952+0000","last_fullsized":"2024-05-04T12:18:23.835952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:59.977088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779675+0000","last_change":"2024-05-04T12:18:21.838819+0000","last_active":"2024-05-04T12:18:24.779675+0000","last_peered":"2024-05-04T12:18:24.779675+0000","last_clean":"2024-05-04T12:18:24.779675+0000","last_became_active":"2024-05-04T12:18:21.838503+0000","last_became_peered":"2024-05-04T12:18:21.838503+0000","last_unstale":"2024-05-04T12:18:24.779675+0000","last_undegraded":"2024-05-04T12:18:24.779675+0000","last_fullsized":"2024-05-04T12:18:24.779675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247583+0000","last_change":"2024-05-04T12:18:22.854643+0000","last_active":"2024-05-04T12:18:24.247583+0000","last_peered":"2024-05-04T12:18:24.247583+0000","last_clean":"2024-05-04T12:18:24.247583+0000","last_became_active":"2024-05-04T12:18:22.854063+0000","last_became_peered":"2024-05-04T12:18:22.854063+0000","last_unstale":"2024-05-04T12:18:24.247583+0000","last_undegraded":"2024-05-04T12:18:24.247583+0000","last_fullsized":"2024-05-04T12:18:24.247583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777273+0000","last_change":"2024-05-04T12:18:21.849103+0000","last_active":"2024-05-04T12:18:24.777273+0000","last_peered":"2024-05-04T12:18:24.777273+0000","last_clean":"2024-05-04T12:18:24.777273+0000","last_became_active":"2024-05-04T12:18:21.848433+0000","last_became_peered":"2024-05-04T12:18:21.848433+0000","last_unstale":"2024-05-04T12:18:24.777273+0000","last_undegraded":"2024-05-04T12:18:24.777273+0000","last_fullsized":"2024-05-04T12:18:24.777273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778407+0000","last_change":"2024-05-04T12:18:21.851455+0000","last_active":"2024-05-04T12:18:24.778407+0000","last_peered":"2024-05-04T12:18:24.778407+0000","last_clean":"2024-05-04T12:18:24.778407+0000","last_became_active":"2024-05-04T12:18:21.851229+0000","last_became_peered":"2024-05-04T12:18:21.851229+0000","last_unstale":"2024-05-04T12:18:24.778407+0000","last_undegraded":"2024-05-04T12:18:24.778407+0000","last_fullsized":"2024-05-04T12:18:24.778407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778619+0000","last_change":"2024-05-04T12:18:22.874715+0000","last_active":"2024-05-04T12:18:24.778619+0000","last_peered":"2024-05-04T12:18:24.778619+0000","last_clean":"2024-05-04T12:18:24.778619+0000","last_became_active":"2024-05-04T12:18:22.869353+0000","last_became_peered":"2024-05-04T12:18:22.869353+0000","last_unstale":"2024-05-04T12:18:24.778619+0000","last_undegraded":"2024-05-04T12:18:24.778619+0000","last_fullsized":"2024-05-04T12:18:24.778619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:47:19.879004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835052+0000","last_change":"2024-05-04T12:18:21.832974+0000","last_active":"2024-05-04T12:18:23.835052+0000","last_peered":"2024-05-04T12:18:23.835052+0000","last_clean":"2024-05-04T12:18:23.835052+0000","last_became_active":"2024-05-04T12:18:21.832317+0000","last_became_peered":"2024-05-04T12:18:21.832317+0000","last_unstale":"2024-05-04T12:18:23.835052+0000","last_undegraded":"2024-05-04T12:18:23.835052+0000","last_fullsized":"2024-05-04T12:18:23.835052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831070+0000","last_change":"2024-05-04T12:18:22.871132+0000","last_active":"2024-05-04T12:18:23.831070+0000","last_peered":"2024-05-04T12:18:23.831070+0000","last_clean":"2024-05-04T12:18:23.831070+0000","last_became_active":"2024-05-04T12:18:22.870622+0000","last_became_peered":"2024-05-04T12:18:22.870622+0000","last_unstale":"2024-05-04T12:18:23.831070+0000","last_undegraded":"2024-05-04T12:18:23.831070+0000","last_fullsized":"2024-05-04T12:18:23.831070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:51.416179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777534+0000","last_change":"2024-05-04T12:18:21.846618+0000","last_active":"2024-05-04T12:18:24.777534+0000","last_peered":"2024-05-04T12:18:24.777534+0000","last_clean":"2024-05-04T12:18:24.777534+0000","last_became_active":"2024-05-04T12:18:21.845832+0000","last_became_peered":"2024-05-04T12:18:21.845832+0000","last_unstale":"2024-05-04T12:18:24.777534+0000","last_undegraded":"2024-05-04T12:18:24.777534+0000","last_fullsized":"2024-05-04T12:18:24.777534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836853+0000","last_change":"2024-05-04T12:18:22.848101+0000","last_active":"2024-05-04T12:18:23.836853+0000","last_peered":"2024-05-04T12:18:23.836853+0000","last_clean":"2024-05-04T12:18:23.836853+0000","last_became_active":"2024-05-04T12:18:22.844375+0000","last_became_peered":"2024-05-04T12:18:22.844375+0000","last_unstale":"2024-05-04T12:18:23.836853+0000","last_undegraded":"2024-05-04T12:18:23.836853+0000","last_fullsized":"2024-05-04T12:18:23.836853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:14:21.945963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836972+0000","last_change":"2024-05-04T12:18:21.847419+0000","last_active":"2024-05-04T12:18:23.836972+0000","last_peered":"2024-05-04T12:18:23.836972+0000","last_clean":"2024-05-04T12:18:23.836972+0000","last_became_active":"2024-05-04T12:18:21.846646+0000","last_became_peered":"2024-05-04T12:18:21.846646+0000","last_unstale":"2024-05-04T12:18:23.836972+0000","last_undegraded":"2024-05-04T12:18:23.836972+0000","last_fullsized":"2024-05-04T12:18:23.836972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247246+0000","last_change":"2024-05-04T12:18:22.850137+0000","last_active":"2024-05-04T12:18:24.247246+0000","last_peered":"2024-05-04T12:18:24.247246+0000","last_clean":"2024-05-04T12:18:24.247246+0000","last_became_active":"2024-05-04T12:18:22.849691+0000","last_became_peered":"2024-05-04T12:18:22.849691+0000","last_unstale":"2024-05-04T12:18:24.247246+0000","last_undegraded":"2024-05-04T12:18:24.247246+0000","last_fullsized":"2024-05-04T12:18:24.247246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:27:57.193126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.832204+0000","last_change":"2024-05-04T12:18:21.838666+0000","last_active":"2024-05-04T12:18:23.832204+0000","last_peered":"2024-05-04T12:18:23.832204+0000","last_clean":"2024-05-04T12:18:23.832204+0000","last_became_active":"2024-05-04T12:18:21.838251+0000","last_became_peered":"2024-05-04T12:18:21.838251+0000","last_unstale":"2024-05-04T12:18:23.832204+0000","last_undegraded":"2024-05-04T12:18:23.832204+0000","last_fullsized":"2024-05-04T12:18:23.832204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836726+0000","last_change":"2024-05-04T12:18:22.849782+0000","last_active":"2024-05-04T12:18:23.836726+0000","last_peered":"2024-05-04T12:18:23.836726+0000","last_clean":"2024-05-04T12:18:23.836726+0000","last_became_active":"2024-05-04T12:18:22.843535+0000","last_became_peered":"2024-05-04T12:18:22.843535+0000","last_unstale":"2024-05-04T12:18:23.836726+0000","last_undegraded":"2024-05-04T12:18:23.836726+0000","last_fullsized":"2024-05-04T12:18:23.836726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:40.494337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777423+0000","last_change":"2024-05-04T12:18:21.844876+0000","last_active":"2024-05-04T12:18:24.777423+0000","last_peered":"2024-05-04T12:18:24.777423+0000","last_clean":"2024-05-04T12:18:24.777423+0000","last_became_active":"2024-05-04T12:18:21.844036+0000","last_became_peered":"2024-05-04T12:18:21.844036+0000","last_unstale":"2024-05-04T12:18:24.777423+0000","last_undegraded":"2024-05-04T12:18:24.777423+0000","last_fullsized":"2024-05-04T12:18:24.777423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297461+0000","last_change":"2024-05-04T12:18:22.878315+0000","last_active":"2024-05-04T12:18:24.297461+0000","last_peered":"2024-05-04T12:18:24.297461+0000","last_clean":"2024-05-04T12:18:24.297461+0000","last_became_active":"2024-05-04T12:18:22.877603+0000","last_became_peered":"2024-05-04T12:18:22.877603+0000","last_unstale":"2024-05-04T12:18:24.297461+0000","last_undegraded":"2024-05-04T12:18:24.297461+0000","last_fullsized":"2024-05-04T12:18:24.297461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:14:21.945963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247649+0000","last_change":"2024-05-04T12:18:21.829283+0000","last_active":"2024-05-04T12:18:24.247649+0000","last_peered":"2024-05-04T12:18:24.247649+0000","last_clean":"2024-05-04T12:18:24.247649+0000","last_became_active":"2024-05-04T12:18:21.828821+0000","last_became_peered":"2024-05-04T12:18:21.828821+0000","last_unstale":"2024-05-04T12:18:24.247649+0000","last_undegraded":"2024-05-04T12:18:24.247649+0000","last_fullsized":"2024-05-04T12:18:24.247649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:27:35.940816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247296+0000","last_change":"2024-05-04T12:18:22.851641+0000","last_active":"2024-05-04T12:18:24.247296+0000","last_peered":"2024-05-04T12:18:24.247296+0000","last_clean":"2024-05-04T12:18:24.247296+0000","last_became_active":"2024-05-04T12:18:22.850896+0000","last_became_peered":"2024-05-04T12:18:22.850896+0000","last_unstale":"2024-05-04T12:18:24.247296+0000","last_undegraded":"2024-05-04T12:18:24.247296+0000","last_fullsized":"2024-05-04T12:18:24.247296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:18:08.570317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836170+0000","last_change":"2024-05-04T12:18:21.837372+0000","last_active":"2024-05-04T12:18:23.836170+0000","last_peered":"2024-05-04T12:18:23.836170+0000","last_clean":"2024-05-04T12:18:23.836170+0000","last_became_active":"2024-05-04T12:18:21.836808+0000","last_became_peered":"2024-05-04T12:18:21.836808+0000","last_unstale":"2024-05-04T12:18:23.836170+0000","last_undegraded":"2024-05-04T12:18:23.836170+0000","last_fullsized":"2024-05-04T12:18:23.836170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297317+0000","last_change":"2024-05-04T12:18:22.852179+0000","last_active":"2024-05-04T12:18:24.297317+0000","last_peered":"2024-05-04T12:18:24.297317+0000","last_clean":"2024-05-04T12:18:24.297317+0000","last_became_active":"2024-05-04T12:18:22.851865+0000","last_became_peered":"2024-05-04T12:18:22.851865+0000","last_unstale":"2024-05-04T12:18:24.297317+0000","last_undegraded":"2024-05-04T12:18:24.297317+0000","last_fullsized":"2024-05-04T12:18:24.297317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:28:22.639254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.776743+0000","last_change":"2024-05-04T12:18:21.846060+0000","last_active":"2024-05-04T12:18:24.776743+0000","last_peered":"2024-05-04T12:18:24.776743+0000","last_clean":"2024-05-04T12:18:24.776743+0000","last_became_active":"2024-05-04T12:18:21.844968+0000","last_became_peered":"2024-05-04T12:18:21.844968+0000","last_unstale":"2024-05-04T12:18:24.776743+0000","last_undegraded":"2024-05-04T12:18:24.776743+0000","last_fullsized":"2024-05-04T12:18:24.776743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777104+0000","last_change":"2024-05-04T12:18:22.870884+0000","last_active":"2024-05-04T12:18:24.777104+0000","last_peered":"2024-05-04T12:18:24.777104+0000","last_clean":"2024-05-04T12:18:24.777104+0000","last_became_active":"2024-05-04T12:18:22.870454+0000","last_became_peered":"2024-05-04T12:18:22.870454+0000","last_unstale":"2024-05-04T12:18:24.777104+0000","last_undegraded":"2024-05-04T12:18:24.777104+0000","last_fullsized":"2024-05-04T12:18:24.777104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:47:19.879004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777408+0000","last_change":"2024-05-04T12:18:21.831214+0000","last_active":"2024-05-04T12:18:24.777408+0000","last_peered":"2024-05-04T12:18:24.777408+0000","last_clean":"2024-05-04T12:18:24.777408+0000","last_became_active":"2024-05-04T12:18:21.830771+0000","last_became_peered":"2024-05-04T12:18:21.830771+0000","last_unstale":"2024-05-04T12:18:24.777408+0000","last_undegraded":"2024-05-04T12:18:24.777408+0000","last_fullsized":"2024-05-04T12:18:24.777408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:27:35.940816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296985+0000","last_change":"2024-05-04T12:18:22.852135+0000","last_active":"2024-05-04T12:18:24.296985+0000","last_peered":"2024-05-04T12:18:24.296985+0000","last_clean":"2024-05-04T12:18:24.296985+0000","last_became_active":"2024-05-04T12:18:22.851686+0000","last_became_peered":"2024-05-04T12:18:22.851686+0000","last_unstale":"2024-05-04T12:18:24.296985+0000","last_undegraded":"2024-05-04T12:18:24.296985+0000","last_fullsized":"2024-05-04T12:18:24.296985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:40:13.207623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836318+0000","last_change":"2024-05-04T12:18:21.830621+0000","last_active":"2024-05-04T12:18:23.836318+0000","last_peered":"2024-05-04T12:18:23.836318+0000","last_clean":"2024-05-04T12:18:23.836318+0000","last_became_active":"2024-05-04T12:18:21.829681+0000","last_became_peered":"2024-05-04T12:18:21.829681+0000","last_unstale":"2024-05-04T12:18:23.836318+0000","last_undegraded":"2024-05-04T12:18:23.836318+0000","last_fullsized":"2024-05-04T12:18:23.836318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:04:24.159320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297861+0000","last_change":"2024-05-04T12:18:22.855366+0000","last_active":"2024-05-04T12:18:24.297861+0000","last_peered":"2024-05-04T12:18:24.297861+0000","last_clean":"2024-05-04T12:18:24.297861+0000","last_became_active":"2024-05-04T12:18:22.855103+0000","last_became_peered":"2024-05-04T12:18:22.855103+0000","last_unstale":"2024-05-04T12:18:24.297861+0000","last_undegraded":"2024-05-04T12:18:24.297861+0000","last_fullsized":"2024-05-04T12:18:24.297861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:00:25.310837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778378+0000","last_change":"2024-05-04T12:18:21.832957+0000","last_active":"2024-05-04T12:18:24.778378+0000","last_peered":"2024-05-04T12:18:24.778378+0000","last_clean":"2024-05-04T12:18:24.778378+0000","last_became_active":"2024-05-04T12:18:21.832274+0000","last_became_peered":"2024-05-04T12:18:21.832274+0000","last_unstale":"2024-05-04T12:18:24.778378+0000","last_undegraded":"2024-05-04T12:18:24.778378+0000","last_fullsized":"2024-05-04T12:18:24.778378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298277+0000","last_change":"2024-05-04T12:18:22.861628+0000","last_active":"2024-05-04T12:18:24.298277+0000","last_peered":"2024-05-04T12:18:24.298277+0000","last_clean":"2024-05-04T12:18:24.298277+0000","last_became_active":"2024-05-04T12:18:22.861348+0000","last_became_peered":"2024-05-04T12:18:22.861348+0000","last_unstale":"2024-05-04T12:18:24.298277+0000","last_undegraded":"2024-05-04T12:18:24.298277+0000","last_fullsized":"2024-05-04T12:18:24.298277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:21:56.962539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.832228+0000","last_change":"2024-05-04T12:18:21.832502+0000","last_active":"2024-05-04T12:18:23.832228+0000","last_peered":"2024-05-04T12:18:23.832228+0000","last_clean":"2024-05-04T12:18:23.832228+0000","last_became_active":"2024-05-04T12:18:21.832085+0000","last_became_peered":"2024-05-04T12:18:21.832085+0000","last_unstale":"2024-05-04T12:18:23.832228+0000","last_undegraded":"2024-05-04T12:18:23.832228+0000","last_fullsized":"2024-05-04T12:18:23.832228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:39.485391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836949+0000","last_change":"2024-05-04T12:18:22.870045+0000","last_active":"2024-05-04T12:18:23.836949+0000","last_peered":"2024-05-04T12:18:23.836949+0000","last_clean":"2024-05-04T12:18:23.836949+0000","last_became_active":"2024-05-04T12:18:22.869646+0000","last_became_peered":"2024-05-04T12:18:22.869646+0000","last_unstale":"2024-05-04T12:18:23.836949+0000","last_undegraded":"2024-05-04T12:18:23.836949+0000","last_fullsized":"2024-05-04T12:18:23.836949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:02:37.747970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777797+0000","last_change":"2024-05-04T12:18:21.839604+0000","last_active":"2024-05-04T12:18:24.777797+0000","last_peered":"2024-05-04T12:18:24.777797+0000","last_clean":"2024-05-04T12:18:24.777797+0000","last_became_active":"2024-05-04T12:18:21.839304+0000","last_became_peered":"2024-05-04T12:18:21.839304+0000","last_unstale":"2024-05-04T12:18:24.777797+0000","last_undegraded":"2024-05-04T12:18:24.777797+0000","last_fullsized":"2024-05-04T12:18:24.777797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297752+0000","last_change":"2024-05-04T12:18:22.854672+0000","last_active":"2024-05-04T12:18:24.297752+0000","last_peered":"2024-05-04T12:18:24.297752+0000","last_clean":"2024-05-04T12:18:24.297752+0000","last_became_active":"2024-05-04T12:18:22.852798+0000","last_became_peered":"2024-05-04T12:18:22.852798+0000","last_unstale":"2024-05-04T12:18:24.297752+0000","last_undegraded":"2024-05-04T12:18:24.297752+0000","last_fullsized":"2024-05-04T12:18:24.297752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:16:56.544684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296796+0000","last_change":"2024-05-04T12:18:12.644577+0000","last_active":"2024-05-04T12:18:24.296796+0000","last_peered":"2024-05-04T12:18:24.296796+0000","last_clean":"2024-05-04T12:18:24.296796+0000","last_became_active":"2024-05-04T12:18:12.640041+0000","last_became_peered":"2024-05-04T12:18:12.640041+0000","last_unstale":"2024-05-04T12:18:24.296796+0000","last_undegraded":"2024-05-04T12:18:24.296796+0000","last_fullsized":"2024-05-04T12:18:24.296796+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_clean_scrub_stamp":"2024-05-04T12:18:11.260157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:15:02.089387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836180+0000","last_change":"2024-05-04T12:18:21.837361+0000","last_active":"2024-05-04T12:18:23.836180+0000","last_peered":"2024-05-04T12:18:23.836180+0000","last_clean":"2024-05-04T12:18:23.836180+0000","last_became_active":"2024-05-04T12:18:21.836437+0000","last_became_peered":"2024-05-04T12:18:21.836437+0000","last_unstale":"2024-05-04T12:18:23.836180+0000","last_undegraded":"2024-05-04T12:18:23.836180+0000","last_fullsized":"2024-05-04T12:18:23.836180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:48:12.147734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779602+0000","last_change":"2024-05-04T12:18:22.868591+0000","last_active":"2024-05-04T12:18:24.779602+0000","last_peered":"2024-05-04T12:18:24.779602+0000","last_clean":"2024-05-04T12:18:24.779602+0000","last_became_active":"2024-05-04T12:18:22.868158+0000","last_became_peered":"2024-05-04T12:18:22.868158+0000","last_unstale":"2024-05-04T12:18:24.779602+0000","last_undegraded":"2024-05-04T12:18:24.779602+0000","last_fullsized":"2024-05-04T12:18:24.779602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:05:46.918761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296711+0000","last_change":"2024-05-04T12:18:21.857252+0000","last_active":"2024-05-04T12:18:24.296711+0000","last_peered":"2024-05-04T12:18:24.296711+0000","last_clean":"2024-05-04T12:18:24.296711+0000","last_became_active":"2024-05-04T12:18:21.855446+0000","last_became_peered":"2024-05-04T12:18:21.855446+0000","last_unstale":"2024-05-04T12:18:24.296711+0000","last_undegraded":"2024-05-04T12:18:24.296711+0000","last_fullsized":"2024-05-04T12:18:24.296711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.246868+0000","last_change":"2024-05-04T12:18:22.862302+0000","last_active":"2024-05-04T12:18:24.246868+0000","last_peered":"2024-05-04T12:18:24.246868+0000","last_clean":"2024-05-04T12:18:24.246868+0000","last_became_active":"2024-05-04T12:18:22.859627+0000","last_became_peered":"2024-05-04T12:18:22.859627+0000","last_unstale":"2024-05-04T12:18:24.246868+0000","last_undegraded":"2024-05-04T12:18:24.246868+0000","last_fullsized":"2024-05-04T12:18:24.246868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:16:07.430915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778081+0000","last_change":"2024-05-04T12:18:21.832787+0000","last_active":"2024-05-04T12:18:24.778081+0000","last_peered":"2024-05-04T12:18:24.778081+0000","last_clean":"2024-05-04T12:18:24.778081+0000","last_became_active":"2024-05-04T12:18:21.832247+0000","last_became_peered":"2024-05-04T12:18:21.832247+0000","last_unstale":"2024-05-04T12:18:24.778081+0000","last_undegraded":"2024-05-04T12:18:24.778081+0000","last_fullsized":"2024-05-04T12:18:24.778081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:48:12.147734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831403+0000","last_change":"2024-05-04T12:18:22.863972+0000","last_active":"2024-05-04T12:18:23.831403+0000","last_peered":"2024-05-04T12:18:23.831403+0000","last_clean":"2024-05-04T12:18:23.831403+0000","last_became_active":"2024-05-04T12:18:22.863122+0000","last_became_peered":"2024-05-04T12:18:22.863122+0000","last_unstale":"2024-05-04T12:18:23.831403+0000","last_undegraded":"2024-05-04T12:18:23.831403+0000","last_fullsized":"2024-05-04T12:18:23.831403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:47:19.879004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836266+0000","last_change":"2024-05-04T12:18:21.835740+0000","last_active":"2024-05-04T12:18:23.836266+0000","last_peered":"2024-05-04T12:18:23.836266+0000","last_clean":"2024-05-04T12:18:23.836266+0000","last_became_active":"2024-05-04T12:18:21.834835+0000","last_became_peered":"2024-05-04T12:18:21.834835+0000","last_unstale":"2024-05-04T12:18:23.836266+0000","last_undegraded":"2024-05-04T12:18:23.836266+0000","last_fullsized":"2024-05-04T12:18:23.836266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835634+0000","last_change":"2024-05-04T12:18:22.856686+0000","last_active":"2024-05-04T12:18:23.835634+0000","last_peered":"2024-05-04T12:18:23.835634+0000","last_clean":"2024-05-04T12:18:23.835634+0000","last_became_active":"2024-05-04T12:18:22.855883+0000","last_became_peered":"2024-05-04T12:18:22.855883+0000","last_unstale":"2024-05-04T12:18:23.835634+0000","last_undegraded":"2024-05-04T12:18:23.835634+0000","last_fullsized":"2024-05-04T12:18:23.835634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:19.851400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831728+0000","last_change":"2024-05-04T12:18:21.831578+0000","last_active":"2024-05-04T12:18:23.831728+0000","last_peered":"2024-05-04T12:18:23.831728+0000","last_clean":"2024-05-04T12:18:23.831728+0000","last_became_active":"2024-05-04T12:18:21.830473+0000","last_became_peered":"2024-05-04T12:18:21.830473+0000","last_unstale":"2024-05-04T12:18:23.831728+0000","last_undegraded":"2024-05-04T12:18:23.831728+0000","last_fullsized":"2024-05-04T12:18:23.831728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.837084+0000","last_change":"2024-05-04T12:18:22.878341+0000","last_active":"2024-05-04T12:18:23.837084+0000","last_peered":"2024-05-04T12:18:23.837084+0000","last_clean":"2024-05-04T12:18:23.837084+0000","last_became_active":"2024-05-04T12:18:22.878015+0000","last_became_peered":"2024-05-04T12:18:22.878015+0000","last_unstale":"2024-05-04T12:18:23.837084+0000","last_undegraded":"2024-05-04T12:18:23.837084+0000","last_fullsized":"2024-05-04T12:18:23.837084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:00:25.310837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298025+0000","last_change":"2024-05-04T12:18:21.857646+0000","last_active":"2024-05-04T12:18:24.298025+0000","last_peered":"2024-05-04T12:18:24.298025+0000","last_clean":"2024-05-04T12:18:24.298025+0000","last_became_active":"2024-05-04T12:18:21.856638+0000","last_became_peered":"2024-05-04T12:18:21.856638+0000","last_unstale":"2024-05-04T12:18:24.298025+0000","last_undegraded":"2024-05-04T12:18:24.298025+0000","last_fullsized":"2024-05-04T12:18:24.298025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:09:02.023381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831578+0000","last_change":"2024-05-04T12:18:22.862113+0000","last_active":"2024-05-04T12:18:23.831578+0000","last_peered":"2024-05-04T12:18:23.831578+0000","last_clean":"2024-05-04T12:18:23.831578+0000","last_became_active":"2024-05-04T12:18:22.861682+0000","last_became_peered":"2024-05-04T12:18:22.861682+0000","last_unstale":"2024-05-04T12:18:23.831578+0000","last_undegraded":"2024-05-04T12:18:23.831578+0000","last_fullsized":"2024-05-04T12:18:23.831578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.636826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.775560+0000","last_change":"2024-05-04T12:18:21.846869+0000","last_active":"2024-05-04T12:18:24.775560+0000","last_peered":"2024-05-04T12:18:24.775560+0000","last_clean":"2024-05-04T12:18:24.775560+0000","last_became_active":"2024-05-04T12:18:21.846514+0000","last_became_peered":"2024-05-04T12:18:21.846514+0000","last_unstale":"2024-05-04T12:18:24.775560+0000","last_undegraded":"2024-05-04T12:18:24.775560+0000","last_fullsized":"2024-05-04T12:18:24.775560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779211+0000","last_change":"2024-05-04T12:18:22.879784+0000","last_active":"2024-05-04T12:18:24.779211+0000","last_peered":"2024-05-04T12:18:24.779211+0000","last_clean":"2024-05-04T12:18:24.779211+0000","last_became_active":"2024-05-04T12:18:22.877733+0000","last_became_peered":"2024-05-04T12:18:22.877733+0000","last_unstale":"2024-05-04T12:18:24.779211+0000","last_undegraded":"2024-05-04T12:18:24.779211+0000","last_fullsized":"2024-05-04T12:18:24.779211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:10:47.797595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777681+0000","last_change":"2024-05-04T12:18:21.854368+0000","last_active":"2024-05-04T12:18:24.777681+0000","last_peered":"2024-05-04T12:18:24.777681+0000","last_clean":"2024-05-04T12:18:24.777681+0000","last_became_active":"2024-05-04T12:18:21.852721+0000","last_became_peered":"2024-05-04T12:18:21.852721+0000","last_unstale":"2024-05-04T12:18:24.777681+0000","last_undegraded":"2024-05-04T12:18:24.777681+0000","last_fullsized":"2024-05-04T12:18:24.777681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247454+0000","last_change":"2024-05-04T12:18:22.853292+0000","last_active":"2024-05-04T12:18:24.247454+0000","last_peered":"2024-05-04T12:18:24.247454+0000","last_clean":"2024-05-04T12:18:24.247454+0000","last_became_active":"2024-05-04T12:18:22.852401+0000","last_became_peered":"2024-05-04T12:18:22.852401+0000","last_unstale":"2024-05-04T12:18:24.247454+0000","last_undegraded":"2024-05-04T12:18:24.247454+0000","last_fullsized":"2024-05-04T12:18:24.247454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296652+0000","last_change":"2024-05-04T12:18:21.858135+0000","last_active":"2024-05-04T12:18:24.296652+0000","last_peered":"2024-05-04T12:18:24.296652+0000","last_clean":"2024-05-04T12:18:24.296652+0000","last_became_active":"2024-05-04T12:18:21.856430+0000","last_became_peered":"2024-05-04T12:18:21.856430+0000","last_unstale":"2024-05-04T12:18:24.296652+0000","last_undegraded":"2024-05-04T12:18:24.296652+0000","last_fullsized":"2024-05-04T12:18:24.296652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835773+0000","last_change":"2024-05-04T12:18:22.852562+0000","last_active":"2024-05-04T12:18:23.835773+0000","last_peered":"2024-05-04T12:18:23.835773+0000","last_clean":"2024-05-04T12:18:23.835773+0000","last_became_active":"2024-05-04T12:18:22.851980+0000","last_became_peered":"2024-05-04T12:18:22.851980+0000","last_unstale":"2024-05-04T12:18:23.835773+0000","last_undegraded":"2024-05-04T12:18:23.835773+0000","last_fullsized":"2024-05-04T12:18:23.835773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:23.487598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831190+0000","last_change":"2024-05-04T12:18:21.837963+0000","last_active":"2024-05-04T12:18:23.831190+0000","last_peered":"2024-05-04T12:18:23.831190+0000","last_clean":"2024-05-04T12:18:23.831190+0000","last_became_active":"2024-05-04T12:18:21.837563+0000","last_became_peered":"2024-05-04T12:18:21.837563+0000","last_unstale":"2024-05-04T12:18:23.831190+0000","last_undegraded":"2024-05-04T12:18:23.831190+0000","last_fullsized":"2024-05-04T12:18:23.831190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778827+0000","last_change":"2024-05-04T12:18:22.874277+0000","last_active":"2024-05-04T12:18:24.778827+0000","last_peered":"2024-05-04T12:18:24.778827+0000","last_clean":"2024-05-04T12:18:24.778827+0000","last_became_active":"2024-05-04T12:18:22.869298+0000","last_became_peered":"2024-05-04T12:18:22.869298+0000","last_unstale":"2024-05-04T12:18:24.778827+0000","last_undegraded":"2024-05-04T12:18:24.778827+0000","last_fullsized":"2024-05-04T12:18:24.778827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831728+0000","last_change":"2024-05-04T12:18:21.831177+0000","last_active":"2024-05-04T12:18:23.831728+0000","last_peered":"2024-05-04T12:18:23.831728+0000","last_clean":"2024-05-04T12:18:23.831728+0000","last_became_active":"2024-05-04T12:18:21.830570+0000","last_became_peered":"2024-05-04T12:18:21.830570+0000","last_unstale":"2024-05-04T12:18:23.831728+0000","last_undegraded":"2024-05-04T12:18:23.831728+0000","last_fullsized":"2024-05-04T12:18:23.831728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779529+0000","last_change":"2024-05-04T12:18:22.880190+0000","last_active":"2024-05-04T12:18:24.779529+0000","last_peered":"2024-05-04T12:18:24.779529+0000","last_clean":"2024-05-04T12:18:24.779529+0000","last_became_active":"2024-05-04T12:18:22.879801+0000","last_became_peered":"2024-05-04T12:18:22.879801+0000","last_unstale":"2024-05-04T12:18:24.779529+0000","last_undegraded":"2024-05-04T12:18:24.779529+0000","last_fullsized":"2024-05-04T12:18:24.779529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:40:13.207623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297558+0000","last_change":"2024-05-04T12:18:21.860830+0000","last_active":"2024-05-04T12:18:24.297558+0000","last_peered":"2024-05-04T12:18:24.297558+0000","last_clean":"2024-05-04T12:18:24.297558+0000","last_became_active":"2024-05-04T12:18:21.860156+0000","last_became_peered":"2024-05-04T12:18:21.860156+0000","last_unstale":"2024-05-04T12:18:24.297558+0000","last_undegraded":"2024-05-04T12:18:24.297558+0000","last_fullsized":"2024-05-04T12:18:24.297558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:39.485391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.776930+0000","last_change":"2024-05-04T12:18:22.860205+0000","last_active":"2024-05-04T12:18:24.776930+0000","last_peered":"2024-05-04T12:18:24.776930+0000","last_clean":"2024-05-04T12:18:24.776930+0000","last_became_active":"2024-05-04T12:18:22.859825+0000","last_became_peered":"2024-05-04T12:18:22.859825+0000","last_unstale":"2024-05-04T12:18:24.776930+0000","last_undegraded":"2024-05-04T12:18:24.776930+0000","last_fullsized":"2024-05-04T12:18:24.776930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:18:08.570317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.837078+0000","last_change":"2024-05-04T12:18:22.844339+0000","last_active":"2024-05-04T12:18:23.837078+0000","last_peered":"2024-05-04T12:18:23.837078+0000","last_clean":"2024-05-04T12:18:23.837078+0000","last_became_active":"2024-05-04T12:18:22.843807+0000","last_became_peered":"2024-05-04T12:18:22.843807+0000","last_unstale":"2024-05-04T12:18:23.837078+0000","last_undegraded":"2024-05-04T12:18:23.837078+0000","last_fullsized":"2024-05-04T12:18:23.837078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:16:56.544684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297584+0000","last_change":"2024-05-04T12:18:21.860670+0000","last_active":"2024-05-04T12:18:24.297584+0000","last_peered":"2024-05-04T12:18:24.297584+0000","last_clean":"2024-05-04T12:18:24.297584+0000","last_became_active":"2024-05-04T12:18:21.859826+0000","last_became_peered":"2024-05-04T12:18:21.859826+0000","last_unstale":"2024-05-04T12:18:24.297584+0000","last_undegraded":"2024-05-04T12:18:24.297584+0000","last_fullsized":"2024-05-04T12:18:24.297584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836737+0000","last_change":"2024-05-04T12:18:22.846218+0000","last_active":"2024-05-04T12:18:23.836737+0000","last_peered":"2024-05-04T12:18:23.836737+0000","last_clean":"2024-05-04T12:18:23.836737+0000","last_became_active":"2024-05-04T12:18:22.845967+0000","last_became_peered":"2024-05-04T12:18:22.845967+0000","last_unstale":"2024-05-04T12:18:23.836737+0000","last_undegraded":"2024-05-04T12:18:23.836737+0000","last_fullsized":"2024-05-04T12:18:23.836737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:05:46.918761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297709+0000","last_change":"2024-05-04T12:18:21.860069+0000","last_active":"2024-05-04T12:18:24.297709+0000","last_peered":"2024-05-04T12:18:24.297709+0000","last_clean":"2024-05-04T12:18:24.297709+0000","last_became_active":"2024-05-04T12:18:21.859583+0000","last_became_peered":"2024-05-04T12:18:21.859583+0000","last_unstale":"2024-05-04T12:18:24.297709+0000","last_undegraded":"2024-05-04T12:18:24.297709+0000","last_fullsized":"2024-05-04T12:18:24.297709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:03:35.131163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247407+0000","last_change":"2024-05-04T12:18:22.856277+0000","last_active":"2024-05-04T12:18:24.247407+0000","last_peered":"2024-05-04T12:18:24.247407+0000","last_clean":"2024-05-04T12:18:24.247407+0000","last_became_active":"2024-05-04T12:18:22.855958+0000","last_became_peered":"2024-05-04T12:18:22.855958+0000","last_unstale":"2024-05-04T12:18:24.247407+0000","last_undegraded":"2024-05-04T12:18:24.247407+0000","last_fullsized":"2024-05-04T12:18:24.247407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.636826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779181+0000","last_change":"2024-05-04T12:18:21.835661+0000","last_active":"2024-05-04T12:18:24.779181+0000","last_peered":"2024-05-04T12:18:24.779181+0000","last_clean":"2024-05-04T12:18:24.779181+0000","last_became_active":"2024-05-04T12:18:21.835299+0000","last_became_peered":"2024-05-04T12:18:21.835299+0000","last_unstale":"2024-05-04T12:18:24.779181+0000","last_undegraded":"2024-05-04T12:18:24.779181+0000","last_fullsized":"2024-05-04T12:18:24.779181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:09:02.023381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779181+0000","last_change":"2024-05-04T12:18:22.878092+0000","last_active":"2024-05-04T12:18:24.779181+0000","last_peered":"2024-05-04T12:18:24.779181+0000","last_clean":"2024-05-04T12:18:24.779181+0000","last_became_active":"2024-05-04T12:18:22.875650+0000","last_became_peered":"2024-05-04T12:18:22.875650+0000","last_unstale":"2024-05-04T12:18:24.779181+0000","last_undegraded":"2024-05-04T12:18:24.779181+0000","last_fullsized":"2024-05-04T12:18:24.779181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247663+0000","last_change":"2024-05-04T12:18:21.847981+0000","last_active":"2024-05-04T12:18:24.247663+0000","last_peered":"2024-05-04T12:18:24.247663+0000","last_clean":"2024-05-04T12:18:24.247663+0000","last_became_active":"2024-05-04T12:18:21.847556+0000","last_became_peered":"2024-05-04T12:18:21.847556+0000","last_unstale":"2024-05-04T12:18:24.247663+0000","last_undegraded":"2024-05-04T12:18:24.247663+0000","last_fullsized":"2024-05-04T12:18:24.247663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:07:48.655726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T12:18:29.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-04T12:18:29.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.778+0000 7f89628c6640 1 Processor -- start 2024-05-04T12:18:29.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.778+0000 7f89628c6640 1 -- start start 2024-05-04T12:18:29.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f89628c6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f895c0cd5e0 0x7f895c06be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:29.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f89628c6640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f895c070de0 0x7f895c0cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:29.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f89628c6640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f895c0b07c0 con 0x7f895c0b1260 2024-05-04T12:18:29.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f89628c6640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f895c0cd9c0 con 0x7f895c0cd5e0 2024-05-04T12:18:29.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f89628c6640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f895c15e8e0 con 0x7f895c070de0 2024-05-04T12:18:29.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f895bfff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f895c0cd5e0 0x7f895c06be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:29.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f895bfff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f895c0cd5e0 0x7f895c06be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53628/0 (socket says 172.21.15.52:53628) 2024-05-04T12:18:29.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f895bfff640 1 -- 172.21.15.52:0/3436651079 learned_addr learned my addr 172.21.15.52:0/3436651079 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:29.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f8960e63640 1 --2- 172.21.15.52:0/3436651079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f895c070de0 0x7f895c0cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:29.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f895bfff640 1 -- 172.21.15.52:0/3436651079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f895c070de0 msgr2=0x7f895c0cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:29.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f895bfff640 1 --2- 172.21.15.52:0/3436651079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f895c070de0 0x7f895c0cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f895bfff640 1 -- 172.21.15.52:0/3436651079 >> v1:172.21.15.111:6789/0 conn(0x7f895c0b1260 legacy=0x7f895c0706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:29.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f895bfff640 1 -- 172.21.15.52:0/3436651079 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895c0cfc50 con 0x7f895c0cd5e0 2024-05-04T12:18:29.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f895bfff640 1 --2- 172.21.15.52:0/3436651079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f895c0cd5e0 0x7f895c06be80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f894c009870 tx=0x7f894c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3b741dc99ca3fd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:29.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f895affd640 1 -- 172.21.15.52:0/3436651079 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c0aa610 con 0x7f895c0cd5e0 2024-05-04T12:18:29.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f895affd640 1 -- 172.21.15.52:0/3436651079 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f894c0af070 con 0x7f895c0cd5e0 2024-05-04T12:18:29.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.782+0000 7f895affd640 1 -- 172.21.15.52:0/3436651079 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c0aa9d0 con 0x7f895c0cd5e0 2024-05-04T12:18:29.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 -- 172.21.15.52:0/3436651079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f895c0cd5e0 msgr2=0x7f895c06be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 --2- 172.21.15.52:0/3436651079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f895c0cd5e0 0x7f895c06be80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f894c009870 tx=0x7f894c01c4f0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 -- 172.21.15.52:0/3436651079 shutdown_connections 2024-05-04T12:18:29.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 --2- 172.21.15.52:0/3436651079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f895c070de0 0x7f895c0cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 --2- 172.21.15.52:0/3436651079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f895c0cd5e0 0x7f895c06be80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 -- 172.21.15.52:0/3436651079 >> 172.21.15.52:0/3436651079 conn(0x7f895c0b2380 msgr2=0x7f895c0b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:29.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 -- 172.21.15.52:0/3436651079 shutdown_connections 2024-05-04T12:18:29.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 -- 172.21.15.52:0/3436651079 wait complete. 2024-05-04T12:18:29.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 Processor -- start 2024-05-04T12:18:29.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 -- start start 2024-05-04T12:18:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f895c070de0 0x7f895c1fc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f895c0b1260 0x7f895c15d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f895c0cd5e0 0x7f895c160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f895c0cc8c0 con 0x7f895c0cd5e0 2024-05-04T12:18:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f895c0b1f20 con 0x7f895c0b1260 2024-05-04T12:18:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f895c0cf8e0 con 0x7f895c070de0 2024-05-04T12:18:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f8960e63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f895c0cd5e0 0x7f895c160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f8960e63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f895c0cd5e0 0x7f895c160cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53638/0 (socket says 172.21.15.52:53638) 2024-05-04T12:18:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f8960e63640 1 -- 172.21.15.52:0/1091018427 learned_addr learned my addr 172.21.15.52:0/1091018427 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f895bfff640 1 --2- 172.21.15.52:0/1091018427 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f895c070de0 0x7f895c1fc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f895b7fe640 1 --2- 172.21.15.52:0/1091018427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f895c0b1260 0x7f895c15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f8960e63640 1 -- 172.21.15.52:0/1091018427 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f895c070de0 msgr2=0x7f895c1fc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f8960e63640 1 --2- 172.21.15.52:0/1091018427 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f895c070de0 0x7f895c1fc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f8960e63640 1 -- 172.21.15.52:0/1091018427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f895c0b1260 msgr2=0x7f895c15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f8960e63640 1 --2- 172.21.15.52:0/1091018427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f895c0b1260 0x7f895c15d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f8960e63640 1 -- 172.21.15.52:0/1091018427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895c1fc820 con 0x7f895c0cd5e0 2024-05-04T12:18:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f8960e63640 1 --2- 172.21.15.52:0/1091018427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f895c0cd5e0 0x7f895c160cb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f895400d800 tx=0x7f895400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f8958ff9640 1 -- 172.21.15.52:0/1091018427 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895400b6c0 con 0x7f895c0cd5e0 2024-05-04T12:18:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f8958ff9640 1 -- 172.21.15.52:0/1091018427 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8954019070 con 0x7f895c0cd5e0 2024-05-04T12:18:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f8958ff9640 1 -- 172.21.15.52:0/1091018427 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895400baa0 con 0x7f895c0cd5e0 2024-05-04T12:18:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 -- 172.21.15.52:0/1091018427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895c157160 con 0x7f895c0cd5e0 2024-05-04T12:18:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.786+0000 7f89628c6640 1 -- 172.21.15.52:0/1091018427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895c1576e0 con 0x7f895c0cd5e0 2024-05-04T12:18:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.790+0000 7f89628c6640 1 -- 172.21.15.52:0/1091018427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f895c157fd0 con 0x7f895c0cd5e0 2024-05-04T12:18:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.790+0000 7f8958ff9640 1 -- 172.21.15.52:0/1091018427 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f895400bc40 con 0x7f895c0cd5e0 2024-05-04T12:18:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.790+0000 7f8958ff9640 1 --2- 172.21.15.52:0/1091018427 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f89480678c0 0x7f8948069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.790+0000 7f895bfff640 1 --2- 172.21.15.52:0/1091018427 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f89480678c0 0x7f8948069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.790+0000 7f8958ff9640 1 -- 172.21.15.52:0/1091018427 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8954086110 con 0x7f895c0cd5e0 2024-05-04T12:18:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.790+0000 7f895bfff640 1 --2- 172.21.15.52:0/1091018427 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f89480678c0 0x7f8948069d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f894c0ae000 tx=0x7f894c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:29.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.794+0000 7f8958ff9640 1 -- 172.21.15.52:0/1091018427 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f895c1fc820 con 0x7f895c0cd5e0 2024-05-04T12:18:29.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.950+0000 7f89628c6640 1 -- 172.21.15.52:0/1091018427 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f895c062e50 con 0x7f89480678c0 2024-05-04T12:18:29.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.962+0000 7f8958ff9640 1 -- 172.21.15.52:0/1091018427 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251872 (secure 0 0 0) 0x7f895c062e50 con 0x7f89480678c0 2024-05-04T12:18:29.965 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:29.967 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-04T12:18:29.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.962+0000 7f89628c6640 1 -- 172.21.15.52:0/1091018427 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f89480678c0 msgr2=0x7f8948069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.962+0000 7f89628c6640 1 --2- 172.21.15.52:0/1091018427 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f89480678c0 0x7f8948069d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f894c0ae000 tx=0x7f894c0a7000 comp rx=0 tx=0).stop 2024-05-04T12:18:29.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.962+0000 7f89628c6640 1 -- 172.21.15.52:0/1091018427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f895c0cd5e0 msgr2=0x7f895c160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:29.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.962+0000 7f89628c6640 1 --2- 172.21.15.52:0/1091018427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f895c0cd5e0 0x7f895c160cb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f895400d800 tx=0x7f895400dcc0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.962+0000 7f89628c6640 1 -- 172.21.15.52:0/1091018427 shutdown_connections 2024-05-04T12:18:29.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.962+0000 7f89628c6640 1 --2- 172.21.15.52:0/1091018427 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f89480678c0 0x7f8948069d80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.962+0000 7f89628c6640 1 --2- 172.21.15.52:0/1091018427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f895c0cd5e0 0x7f895c160cb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.962+0000 7f89628c6640 1 --2- 172.21.15.52:0/1091018427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f895c0b1260 0x7f895c15d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.962+0000 7f89628c6640 1 --2- 172.21.15.52:0/1091018427 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f895c070de0 0x7f895c1fc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:29.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.962+0000 7f89628c6640 1 -- 172.21.15.52:0/1091018427 >> 172.21.15.52:0/1091018427 conn(0x7f895c0b2380 msgr2=0x7f895c06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:29.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.966+0000 7f89628c6640 1 -- 172.21.15.52:0/1091018427 shutdown_connections 2024-05-04T12:18:29.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:29.966+0000 7f89628c6640 1 -- 172.21.15.52:0/1091018427 wait complete. 2024-05-04T12:18:29.981 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-04T12:18:28.741660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235720,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739000,"statfs":{"total":773094113280,"available":772852736000,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.932093"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298061+0000","last_change":"2024-05-04T12:18:22.859306+0000","last_active":"2024-05-04T12:18:24.298061+0000","last_peered":"2024-05-04T12:18:24.298061+0000","last_clean":"2024-05-04T12:18:24.298061+0000","last_became_active":"2024-05-04T12:18:22.859107+0000","last_became_peered":"2024-05-04T12:18:22.859107+0000","last_unstale":"2024-05-04T12:18:24.298061+0000","last_undegraded":"2024-05-04T12:18:24.298061+0000","last_fullsized":"2024-05-04T12:18:24.298061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:45:17.602528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.832155+0000","last_change":"2024-05-04T12:18:22.848885+0000","last_active":"2024-05-04T12:18:23.832155+0000","last_peered":"2024-05-04T12:18:23.832155+0000","last_clean":"2024-05-04T12:18:23.832155+0000","last_became_active":"2024-05-04T12:18:22.848320+0000","last_became_peered":"2024-05-04T12:18:22.848320+0000","last_unstale":"2024-05-04T12:18:23.832155+0000","last_undegraded":"2024-05-04T12:18:23.832155+0000","last_fullsized":"2024-05-04T12:18:23.832155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247882+0000","last_change":"2024-05-04T12:18:22.861471+0000","last_active":"2024-05-04T12:18:24.247882+0000","last_peered":"2024-05-04T12:18:24.247882+0000","last_clean":"2024-05-04T12:18:24.247882+0000","last_became_active":"2024-05-04T12:18:22.860827+0000","last_became_peered":"2024-05-04T12:18:22.860827+0000","last_unstale":"2024-05-04T12:18:24.247882+0000","last_undegraded":"2024-05-04T12:18:24.247882+0000","last_fullsized":"2024-05-04T12:18:24.247882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:10:47.797595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836758+0000","last_change":"2024-05-04T12:18:22.878704+0000","last_active":"2024-05-04T12:18:23.836758+0000","last_peered":"2024-05-04T12:18:23.836758+0000","last_clean":"2024-05-04T12:18:23.836758+0000","last_became_active":"2024-05-04T12:18:22.878423+0000","last_became_peered":"2024-05-04T12:18:22.878423+0000","last_unstale":"2024-05-04T12:18:23.836758+0000","last_undegraded":"2024-05-04T12:18:23.836758+0000","last_fullsized":"2024-05-04T12:18:23.836758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:28:22.639254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297059+0000","last_change":"2024-05-04T12:18:22.854847+0000","last_active":"2024-05-04T12:18:24.297059+0000","last_peered":"2024-05-04T12:18:24.297059+0000","last_clean":"2024-05-04T12:18:24.297059+0000","last_became_active":"2024-05-04T12:18:22.854509+0000","last_became_peered":"2024-05-04T12:18:22.854509+0000","last_unstale":"2024-05-04T12:18:24.297059+0000","last_undegraded":"2024-05-04T12:18:24.297059+0000","last_fullsized":"2024-05-04T12:18:24.297059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:05:46.918761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835635+0000","last_change":"2024-05-04T12:18:22.856252+0000","last_active":"2024-05-04T12:18:23.835635+0000","last_peered":"2024-05-04T12:18:23.835635+0000","last_clean":"2024-05-04T12:18:23.835635+0000","last_became_active":"2024-05-04T12:18:22.855950+0000","last_became_peered":"2024-05-04T12:18:22.855950+0000","last_unstale":"2024-05-04T12:18:23.835635+0000","last_undegraded":"2024-05-04T12:18:23.835635+0000","last_fullsized":"2024-05-04T12:18:23.835635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:10:47.797595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297649+0000","last_change":"2024-05-04T12:18:22.851916+0000","last_active":"2024-05-04T12:18:24.297649+0000","last_peered":"2024-05-04T12:18:24.297649+0000","last_clean":"2024-05-04T12:18:24.297649+0000","last_became_active":"2024-05-04T12:18:22.851291+0000","last_became_peered":"2024-05-04T12:18:22.851291+0000","last_unstale":"2024-05-04T12:18:24.297649+0000","last_undegraded":"2024-05-04T12:18:24.297649+0000","last_fullsized":"2024-05-04T12:18:24.297649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:02:37.747970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778425+0000","last_change":"2024-05-04T12:18:22.879929+0000","last_active":"2024-05-04T12:18:24.778425+0000","last_peered":"2024-05-04T12:18:24.778425+0000","last_clean":"2024-05-04T12:18:24.778425+0000","last_became_active":"2024-05-04T12:18:22.879364+0000","last_became_peered":"2024-05-04T12:18:22.879364+0000","last_unstale":"2024-05-04T12:18:24.778425+0000","last_undegraded":"2024-05-04T12:18:24.778425+0000","last_fullsized":"2024-05-04T12:18:24.778425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:51.416179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298129+0000","last_change":"2024-05-04T12:18:22.859239+0000","last_active":"2024-05-04T12:18:24.298129+0000","last_peered":"2024-05-04T12:18:24.298129+0000","last_clean":"2024-05-04T12:18:24.298129+0000","last_became_active":"2024-05-04T12:18:22.858831+0000","last_became_peered":"2024-05-04T12:18:22.858831+0000","last_unstale":"2024-05-04T12:18:24.298129+0000","last_undegraded":"2024-05-04T12:18:24.298129+0000","last_fullsized":"2024-05-04T12:18:24.298129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:09:40.457188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777413+0000","last_change":"2024-05-04T12:18:22.853885+0000","last_active":"2024-05-04T12:18:24.777413+0000","last_peered":"2024-05-04T12:18:24.777413+0000","last_clean":"2024-05-04T12:18:24.777413+0000","last_became_active":"2024-05-04T12:18:22.853589+0000","last_became_peered":"2024-05-04T12:18:22.853589+0000","last_unstale":"2024-05-04T12:18:24.777413+0000","last_undegraded":"2024-05-04T12:18:24.777413+0000","last_fullsized":"2024-05-04T12:18:24.777413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777354+0000","last_change":"2024-05-04T12:18:22.856994+0000","last_active":"2024-05-04T12:18:24.777354+0000","last_peered":"2024-05-04T12:18:24.777354+0000","last_clean":"2024-05-04T12:18:24.777354+0000","last_became_active":"2024-05-04T12:18:22.856651+0000","last_became_peered":"2024-05-04T12:18:22.856651+0000","last_unstale":"2024-05-04T12:18:24.777354+0000","last_undegraded":"2024-05-04T12:18:24.777354+0000","last_fullsized":"2024-05-04T12:18:24.777354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836324+0000","last_change":"2024-05-04T12:18:22.864231+0000","last_active":"2024-05-04T12:18:23.836324+0000","last_peered":"2024-05-04T12:18:23.836324+0000","last_clean":"2024-05-04T12:18:23.836324+0000","last_became_active":"2024-05-04T12:18:22.861558+0000","last_became_peered":"2024-05-04T12:18:22.861558+0000","last_unstale":"2024-05-04T12:18:23.836324+0000","last_undegraded":"2024-05-04T12:18:23.836324+0000","last_fullsized":"2024-05-04T12:18:23.836324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:37.729658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779374+0000","last_change":"2024-05-04T12:18:22.868189+0000","last_active":"2024-05-04T12:18:24.779374+0000","last_peered":"2024-05-04T12:18:24.779374+0000","last_clean":"2024-05-04T12:18:24.779374+0000","last_became_active":"2024-05-04T12:18:22.860456+0000","last_became_peered":"2024-05-04T12:18:22.860456+0000","last_unstale":"2024-05-04T12:18:24.779374+0000","last_undegraded":"2024-05-04T12:18:24.779374+0000","last_fullsized":"2024-05-04T12:18:24.779374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:07:22.005194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.246875+0000","last_change":"2024-05-04T12:18:22.853777+0000","last_active":"2024-05-04T12:18:24.246875+0000","last_peered":"2024-05-04T12:18:24.246875+0000","last_clean":"2024-05-04T12:18:24.246875+0000","last_became_active":"2024-05-04T12:18:22.852626+0000","last_became_peered":"2024-05-04T12:18:22.852626+0000","last_unstale":"2024-05-04T12:18:24.246875+0000","last_undegraded":"2024-05-04T12:18:24.246875+0000","last_fullsized":"2024-05-04T12:18:24.246875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.833385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831334+0000","last_change":"2024-05-04T12:18:22.844392+0000","last_active":"2024-05-04T12:18:23.831334+0000","last_peered":"2024-05-04T12:18:23.831334+0000","last_clean":"2024-05-04T12:18:23.831334+0000","last_became_active":"2024-05-04T12:18:22.843539+0000","last_became_peered":"2024-05-04T12:18:22.843539+0000","last_unstale":"2024-05-04T12:18:23.831334+0000","last_undegraded":"2024-05-04T12:18:23.831334+0000","last_fullsized":"2024-05-04T12:18:23.831334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:18:08.570317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.834671+0000","last_change":"2024-05-04T12:18:22.849846+0000","last_active":"2024-05-04T12:18:23.834671+0000","last_peered":"2024-05-04T12:18:23.834671+0000","last_clean":"2024-05-04T12:18:23.834671+0000","last_became_active":"2024-05-04T12:18:22.849552+0000","last_became_peered":"2024-05-04T12:18:22.849552+0000","last_unstale":"2024-05-04T12:18:23.834671+0000","last_undegraded":"2024-05-04T12:18:23.834671+0000","last_fullsized":"2024-05-04T12:18:23.834671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.837479+0000","last_change":"2024-05-04T12:18:22.851494+0000","last_active":"2024-05-04T12:18:23.837479+0000","last_peered":"2024-05-04T12:18:23.837479+0000","last_clean":"2024-05-04T12:18:23.837479+0000","last_became_active":"2024-05-04T12:18:22.851177+0000","last_became_peered":"2024-05-04T12:18:22.851177+0000","last_unstale":"2024-05-04T12:18:23.837479+0000","last_undegraded":"2024-05-04T12:18:23.837479+0000","last_fullsized":"2024-05-04T12:18:23.837479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:52:05.609251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778115+0000","last_change":"2024-05-04T12:18:22.861585+0000","last_active":"2024-05-04T12:18:24.778115+0000","last_peered":"2024-05-04T12:18:24.778115+0000","last_clean":"2024-05-04T12:18:24.778115+0000","last_became_active":"2024-05-04T12:18:22.861304+0000","last_became_peered":"2024-05-04T12:18:22.861304+0000","last_unstale":"2024-05-04T12:18:24.778115+0000","last_undegraded":"2024-05-04T12:18:24.778115+0000","last_fullsized":"2024-05-04T12:18:24.778115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:40.494337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297947+0000","last_change":"2024-05-04T12:18:22.855243+0000","last_active":"2024-05-04T12:18:24.297947+0000","last_peered":"2024-05-04T12:18:24.297947+0000","last_clean":"2024-05-04T12:18:24.297947+0000","last_became_active":"2024-05-04T12:18:22.854938+0000","last_became_peered":"2024-05-04T12:18:22.854938+0000","last_unstale":"2024-05-04T12:18:24.297947+0000","last_undegraded":"2024-05-04T12:18:24.297947+0000","last_fullsized":"2024-05-04T12:18:24.297947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:52:05.609251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777201+0000","last_change":"2024-05-04T12:18:22.870381+0000","last_active":"2024-05-04T12:18:24.777201+0000","last_peered":"2024-05-04T12:18:24.777201+0000","last_clean":"2024-05-04T12:18:24.777201+0000","last_became_active":"2024-05-04T12:18:22.870076+0000","last_became_peered":"2024-05-04T12:18:22.870076+0000","last_unstale":"2024-05-04T12:18:24.777201+0000","last_undegraded":"2024-05-04T12:18:24.777201+0000","last_fullsized":"2024-05-04T12:18:24.777201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.636826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247261+0000","last_change":"2024-05-04T12:18:22.853194+0000","last_active":"2024-05-04T12:18:24.247261+0000","last_peered":"2024-05-04T12:18:24.247261+0000","last_clean":"2024-05-04T12:18:24.247261+0000","last_became_active":"2024-05-04T12:18:22.852625+0000","last_became_peered":"2024-05-04T12:18:22.852625+0000","last_unstale":"2024-05-04T12:18:24.247261+0000","last_undegraded":"2024-05-04T12:18:24.247261+0000","last_fullsized":"2024-05-04T12:18:24.247261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:51.416179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836035+0000","last_change":"2024-05-04T12:18:22.844120+0000","last_active":"2024-05-04T12:18:23.836035+0000","last_peered":"2024-05-04T12:18:23.836035+0000","last_clean":"2024-05-04T12:18:23.836035+0000","last_became_active":"2024-05-04T12:18:22.841231+0000","last_became_peered":"2024-05-04T12:18:22.841231+0000","last_unstale":"2024-05-04T12:18:23.836035+0000","last_undegraded":"2024-05-04T12:18:23.836035+0000","last_fullsized":"2024-05-04T12:18:23.836035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:07:22.005194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778499+0000","last_change":"2024-05-04T12:18:22.877765+0000","last_active":"2024-05-04T12:18:24.778499+0000","last_peered":"2024-05-04T12:18:24.778499+0000","last_clean":"2024-05-04T12:18:24.778499+0000","last_became_active":"2024-05-04T12:18:22.877351+0000","last_became_peered":"2024-05-04T12:18:22.877351+0000","last_unstale":"2024-05-04T12:18:24.778499+0000","last_undegraded":"2024-05-04T12:18:24.778499+0000","last_fullsized":"2024-05-04T12:18:24.778499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:18:08.570317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.834935+0000","last_change":"2024-05-04T12:18:22.878164+0000","last_active":"2024-05-04T12:18:23.834935+0000","last_peered":"2024-05-04T12:18:23.834935+0000","last_clean":"2024-05-04T12:18:23.834935+0000","last_became_active":"2024-05-04T12:18:22.877881+0000","last_became_peered":"2024-05-04T12:18:22.877881+0000","last_unstale":"2024-05-04T12:18:23.834935+0000","last_undegraded":"2024-05-04T12:18:23.834935+0000","last_fullsized":"2024-05-04T12:18:23.834935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778748+0000","last_change":"2024-05-04T12:18:22.880454+0000","last_active":"2024-05-04T12:18:24.778748+0000","last_peered":"2024-05-04T12:18:24.778748+0000","last_clean":"2024-05-04T12:18:24.778748+0000","last_became_active":"2024-05-04T12:18:22.877223+0000","last_became_peered":"2024-05-04T12:18:22.877223+0000","last_unstale":"2024-05-04T12:18:24.778748+0000","last_undegraded":"2024-05-04T12:18:24.778748+0000","last_fullsized":"2024-05-04T12:18:24.778748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.636826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777595+0000","last_change":"2024-05-04T12:18:22.862461+0000","last_active":"2024-05-04T12:18:24.777595+0000","last_peered":"2024-05-04T12:18:24.777595+0000","last_clean":"2024-05-04T12:18:24.777595+0000","last_became_active":"2024-05-04T12:18:22.862122+0000","last_became_peered":"2024-05-04T12:18:22.862122+0000","last_unstale":"2024-05-04T12:18:24.777595+0000","last_undegraded":"2024-05-04T12:18:24.777595+0000","last_fullsized":"2024-05-04T12:18:24.777595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:10:47.797595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831950+0000","last_change":"2024-05-04T12:18:22.870835+0000","last_active":"2024-05-04T12:18:23.831950+0000","last_peered":"2024-05-04T12:18:23.831950+0000","last_clean":"2024-05-04T12:18:23.831950+0000","last_became_active":"2024-05-04T12:18:22.870544+0000","last_became_peered":"2024-05-04T12:18:22.870544+0000","last_unstale":"2024-05-04T12:18:23.831950+0000","last_undegraded":"2024-05-04T12:18:23.831950+0000","last_fullsized":"2024-05-04T12:18:23.831950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779872+0000","last_change":"2024-05-04T12:18:22.869151+0000","last_active":"2024-05-04T12:18:24.779872+0000","last_peered":"2024-05-04T12:18:24.779872+0000","last_clean":"2024-05-04T12:18:24.779872+0000","last_became_active":"2024-05-04T12:18:22.868856+0000","last_became_peered":"2024-05-04T12:18:22.868856+0000","last_unstale":"2024-05-04T12:18:24.779872+0000","last_undegraded":"2024-05-04T12:18:24.779872+0000","last_fullsized":"2024-05-04T12:18:24.779872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:28:22.639254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247351+0000","last_change":"2024-05-04T12:18:22.873982+0000","last_active":"2024-05-04T12:18:24.247351+0000","last_peered":"2024-05-04T12:18:24.247351+0000","last_clean":"2024-05-04T12:18:24.247351+0000","last_became_active":"2024-05-04T12:18:22.871687+0000","last_became_peered":"2024-05-04T12:18:22.871687+0000","last_unstale":"2024-05-04T12:18:24.247351+0000","last_undegraded":"2024-05-04T12:18:24.247351+0000","last_fullsized":"2024-05-04T12:18:24.247351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:47:19.879004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836157+0000","last_change":"2024-05-04T12:18:22.878584+0000","last_active":"2024-05-04T12:18:23.836157+0000","last_peered":"2024-05-04T12:18:23.836157+0000","last_clean":"2024-05-04T12:18:23.836157+0000","last_became_active":"2024-05-04T12:18:22.878093+0000","last_became_peered":"2024-05-04T12:18:22.878093+0000","last_unstale":"2024-05-04T12:18:23.836157+0000","last_undegraded":"2024-05-04T12:18:23.836157+0000","last_fullsized":"2024-05-04T12:18:23.836157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:40:13.207623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777628+0000","last_change":"2024-05-04T12:18:22.851028+0000","last_active":"2024-05-04T12:18:24.777628+0000","last_peered":"2024-05-04T12:18:24.777628+0000","last_clean":"2024-05-04T12:18:24.777628+0000","last_became_active":"2024-05-04T12:18:22.850280+0000","last_became_peered":"2024-05-04T12:18:22.850280+0000","last_unstale":"2024-05-04T12:18:24.777628+0000","last_undegraded":"2024-05-04T12:18:24.777628+0000","last_fullsized":"2024-05-04T12:18:24.777628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:37.729658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.776877+0000","last_change":"2024-05-04T12:18:22.870893+0000","last_active":"2024-05-04T12:18:24.776877+0000","last_peered":"2024-05-04T12:18:24.776877+0000","last_clean":"2024-05-04T12:18:24.776877+0000","last_became_active":"2024-05-04T12:18:22.870517+0000","last_became_peered":"2024-05-04T12:18:22.870517+0000","last_unstale":"2024-05-04T12:18:24.776877+0000","last_undegraded":"2024-05-04T12:18:24.776877+0000","last_fullsized":"2024-05-04T12:18:24.776877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:51.416179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835952+0000","last_change":"2024-05-04T12:18:22.840511+0000","last_active":"2024-05-04T12:18:23.835952+0000","last_peered":"2024-05-04T12:18:23.835952+0000","last_clean":"2024-05-04T12:18:23.835952+0000","last_became_active":"2024-05-04T12:18:22.839811+0000","last_became_peered":"2024-05-04T12:18:22.839811+0000","last_unstale":"2024-05-04T12:18:23.835952+0000","last_undegraded":"2024-05-04T12:18:23.835952+0000","last_fullsized":"2024-05-04T12:18:23.835952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:59.977088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779675+0000","last_change":"2024-05-04T12:18:21.838819+0000","last_active":"2024-05-04T12:18:24.779675+0000","last_peered":"2024-05-04T12:18:24.779675+0000","last_clean":"2024-05-04T12:18:24.779675+0000","last_became_active":"2024-05-04T12:18:21.838503+0000","last_became_peered":"2024-05-04T12:18:21.838503+0000","last_unstale":"2024-05-04T12:18:24.779675+0000","last_undegraded":"2024-05-04T12:18:24.779675+0000","last_fullsized":"2024-05-04T12:18:24.779675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247583+0000","last_change":"2024-05-04T12:18:22.854643+0000","last_active":"2024-05-04T12:18:24.247583+0000","last_peered":"2024-05-04T12:18:24.247583+0000","last_clean":"2024-05-04T12:18:24.247583+0000","last_became_active":"2024-05-04T12:18:22.854063+0000","last_became_peered":"2024-05-04T12:18:22.854063+0000","last_unstale":"2024-05-04T12:18:24.247583+0000","last_undegraded":"2024-05-04T12:18:24.247583+0000","last_fullsized":"2024-05-04T12:18:24.247583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777273+0000","last_change":"2024-05-04T12:18:21.849103+0000","last_active":"2024-05-04T12:18:24.777273+0000","last_peered":"2024-05-04T12:18:24.777273+0000","last_clean":"2024-05-04T12:18:24.777273+0000","last_became_active":"2024-05-04T12:18:21.848433+0000","last_became_peered":"2024-05-04T12:18:21.848433+0000","last_unstale":"2024-05-04T12:18:24.777273+0000","last_undegraded":"2024-05-04T12:18:24.777273+0000","last_fullsized":"2024-05-04T12:18:24.777273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778407+0000","last_change":"2024-05-04T12:18:21.851455+0000","last_active":"2024-05-04T12:18:24.778407+0000","last_peered":"2024-05-04T12:18:24.778407+0000","last_clean":"2024-05-04T12:18:24.778407+0000","last_became_active":"2024-05-04T12:18:21.851229+0000","last_became_peered":"2024-05-04T12:18:21.851229+0000","last_unstale":"2024-05-04T12:18:24.778407+0000","last_undegraded":"2024-05-04T12:18:24.778407+0000","last_fullsized":"2024-05-04T12:18:24.778407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778619+0000","last_change":"2024-05-04T12:18:22.874715+0000","last_active":"2024-05-04T12:18:24.778619+0000","last_peered":"2024-05-04T12:18:24.778619+0000","last_clean":"2024-05-04T12:18:24.778619+0000","last_became_active":"2024-05-04T12:18:22.869353+0000","last_became_peered":"2024-05-04T12:18:22.869353+0000","last_unstale":"2024-05-04T12:18:24.778619+0000","last_undegraded":"2024-05-04T12:18:24.778619+0000","last_fullsized":"2024-05-04T12:18:24.778619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:47:19.879004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835052+0000","last_change":"2024-05-04T12:18:21.832974+0000","last_active":"2024-05-04T12:18:23.835052+0000","last_peered":"2024-05-04T12:18:23.835052+0000","last_clean":"2024-05-04T12:18:23.835052+0000","last_became_active":"2024-05-04T12:18:21.832317+0000","last_became_peered":"2024-05-04T12:18:21.832317+0000","last_unstale":"2024-05-04T12:18:23.835052+0000","last_undegraded":"2024-05-04T12:18:23.835052+0000","last_fullsized":"2024-05-04T12:18:23.835052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831070+0000","last_change":"2024-05-04T12:18:22.871132+0000","last_active":"2024-05-04T12:18:23.831070+0000","last_peered":"2024-05-04T12:18:23.831070+0000","last_clean":"2024-05-04T12:18:23.831070+0000","last_became_active":"2024-05-04T12:18:22.870622+0000","last_became_peered":"2024-05-04T12:18:22.870622+0000","last_unstale":"2024-05-04T12:18:23.831070+0000","last_undegraded":"2024-05-04T12:18:23.831070+0000","last_fullsized":"2024-05-04T12:18:23.831070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:51.416179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777534+0000","last_change":"2024-05-04T12:18:21.846618+0000","last_active":"2024-05-04T12:18:24.777534+0000","last_peered":"2024-05-04T12:18:24.777534+0000","last_clean":"2024-05-04T12:18:24.777534+0000","last_became_active":"2024-05-04T12:18:21.845832+0000","last_became_peered":"2024-05-04T12:18:21.845832+0000","last_unstale":"2024-05-04T12:18:24.777534+0000","last_undegraded":"2024-05-04T12:18:24.777534+0000","last_fullsized":"2024-05-04T12:18:24.777534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836853+0000","last_change":"2024-05-04T12:18:22.848101+0000","last_active":"2024-05-04T12:18:23.836853+0000","last_peered":"2024-05-04T12:18:23.836853+0000","last_clean":"2024-05-04T12:18:23.836853+0000","last_became_active":"2024-05-04T12:18:22.844375+0000","last_became_peered":"2024-05-04T12:18:22.844375+0000","last_unstale":"2024-05-04T12:18:23.836853+0000","last_undegraded":"2024-05-04T12:18:23.836853+0000","last_fullsized":"2024-05-04T12:18:23.836853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:14:21.945963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836972+0000","last_change":"2024-05-04T12:18:21.847419+0000","last_active":"2024-05-04T12:18:23.836972+0000","last_peered":"2024-05-04T12:18:23.836972+0000","last_clean":"2024-05-04T12:18:23.836972+0000","last_became_active":"2024-05-04T12:18:21.846646+0000","last_became_peered":"2024-05-04T12:18:21.846646+0000","last_unstale":"2024-05-04T12:18:23.836972+0000","last_undegraded":"2024-05-04T12:18:23.836972+0000","last_fullsized":"2024-05-04T12:18:23.836972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247246+0000","last_change":"2024-05-04T12:18:22.850137+0000","last_active":"2024-05-04T12:18:24.247246+0000","last_peered":"2024-05-04T12:18:24.247246+0000","last_clean":"2024-05-04T12:18:24.247246+0000","last_became_active":"2024-05-04T12:18:22.849691+0000","last_became_peered":"2024-05-04T12:18:22.849691+0000","last_unstale":"2024-05-04T12:18:24.247246+0000","last_undegraded":"2024-05-04T12:18:24.247246+0000","last_fullsized":"2024-05-04T12:18:24.247246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:27:57.193126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.832204+0000","last_change":"2024-05-04T12:18:21.838666+0000","last_active":"2024-05-04T12:18:23.832204+0000","last_peered":"2024-05-04T12:18:23.832204+0000","last_clean":"2024-05-04T12:18:23.832204+0000","last_became_active":"2024-05-04T12:18:21.838251+0000","last_became_peered":"2024-05-04T12:18:21.838251+0000","last_unstale":"2024-05-04T12:18:23.832204+0000","last_undegraded":"2024-05-04T12:18:23.832204+0000","last_fullsized":"2024-05-04T12:18:23.832204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836726+0000","last_change":"2024-05-04T12:18:22.849782+0000","last_active":"2024-05-04T12:18:23.836726+0000","last_peered":"2024-05-04T12:18:23.836726+0000","last_clean":"2024-05-04T12:18:23.836726+0000","last_became_active":"2024-05-04T12:18:22.843535+0000","last_became_peered":"2024-05-04T12:18:22.843535+0000","last_unstale":"2024-05-04T12:18:23.836726+0000","last_undegraded":"2024-05-04T12:18:23.836726+0000","last_fullsized":"2024-05-04T12:18:23.836726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:40.494337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777423+0000","last_change":"2024-05-04T12:18:21.844876+0000","last_active":"2024-05-04T12:18:24.777423+0000","last_peered":"2024-05-04T12:18:24.777423+0000","last_clean":"2024-05-04T12:18:24.777423+0000","last_became_active":"2024-05-04T12:18:21.844036+0000","last_became_peered":"2024-05-04T12:18:21.844036+0000","last_unstale":"2024-05-04T12:18:24.777423+0000","last_undegraded":"2024-05-04T12:18:24.777423+0000","last_fullsized":"2024-05-04T12:18:24.777423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297461+0000","last_change":"2024-05-04T12:18:22.878315+0000","last_active":"2024-05-04T12:18:24.297461+0000","last_peered":"2024-05-04T12:18:24.297461+0000","last_clean":"2024-05-04T12:18:24.297461+0000","last_became_active":"2024-05-04T12:18:22.877603+0000","last_became_peered":"2024-05-04T12:18:22.877603+0000","last_unstale":"2024-05-04T12:18:24.297461+0000","last_undegraded":"2024-05-04T12:18:24.297461+0000","last_fullsized":"2024-05-04T12:18:24.297461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:14:21.945963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247649+0000","last_change":"2024-05-04T12:18:21.829283+0000","last_active":"2024-05-04T12:18:24.247649+0000","last_peered":"2024-05-04T12:18:24.247649+0000","last_clean":"2024-05-04T12:18:24.247649+0000","last_became_active":"2024-05-04T12:18:21.828821+0000","last_became_peered":"2024-05-04T12:18:21.828821+0000","last_unstale":"2024-05-04T12:18:24.247649+0000","last_undegraded":"2024-05-04T12:18:24.247649+0000","last_fullsized":"2024-05-04T12:18:24.247649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:27:35.940816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247296+0000","last_change":"2024-05-04T12:18:22.851641+0000","last_active":"2024-05-04T12:18:24.247296+0000","last_peered":"2024-05-04T12:18:24.247296+0000","last_clean":"2024-05-04T12:18:24.247296+0000","last_became_active":"2024-05-04T12:18:22.850896+0000","last_became_peered":"2024-05-04T12:18:22.850896+0000","last_unstale":"2024-05-04T12:18:24.247296+0000","last_undegraded":"2024-05-04T12:18:24.247296+0000","last_fullsized":"2024-05-04T12:18:24.247296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:18:08.570317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836170+0000","last_change":"2024-05-04T12:18:21.837372+0000","last_active":"2024-05-04T12:18:23.836170+0000","last_peered":"2024-05-04T12:18:23.836170+0000","last_clean":"2024-05-04T12:18:23.836170+0000","last_became_active":"2024-05-04T12:18:21.836808+0000","last_became_peered":"2024-05-04T12:18:21.836808+0000","last_unstale":"2024-05-04T12:18:23.836170+0000","last_undegraded":"2024-05-04T12:18:23.836170+0000","last_fullsized":"2024-05-04T12:18:23.836170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297317+0000","last_change":"2024-05-04T12:18:22.852179+0000","last_active":"2024-05-04T12:18:24.297317+0000","last_peered":"2024-05-04T12:18:24.297317+0000","last_clean":"2024-05-04T12:18:24.297317+0000","last_became_active":"2024-05-04T12:18:22.851865+0000","last_became_peered":"2024-05-04T12:18:22.851865+0000","last_unstale":"2024-05-04T12:18:24.297317+0000","last_undegraded":"2024-05-04T12:18:24.297317+0000","last_fullsized":"2024-05-04T12:18:24.297317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:28:22.639254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.776743+0000","last_change":"2024-05-04T12:18:21.846060+0000","last_active":"2024-05-04T12:18:24.776743+0000","last_peered":"2024-05-04T12:18:24.776743+0000","last_clean":"2024-05-04T12:18:24.776743+0000","last_became_active":"2024-05-04T12:18:21.844968+0000","last_became_peered":"2024-05-04T12:18:21.844968+0000","last_unstale":"2024-05-04T12:18:24.776743+0000","last_undegraded":"2024-05-04T12:18:24.776743+0000","last_fullsized":"2024-05-04T12:18:24.776743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777104+0000","last_change":"2024-05-04T12:18:22.870884+0000","last_active":"2024-05-04T12:18:24.777104+0000","last_peered":"2024-05-04T12:18:24.777104+0000","last_clean":"2024-05-04T12:18:24.777104+0000","last_became_active":"2024-05-04T12:18:22.870454+0000","last_became_peered":"2024-05-04T12:18:22.870454+0000","last_unstale":"2024-05-04T12:18:24.777104+0000","last_undegraded":"2024-05-04T12:18:24.777104+0000","last_fullsized":"2024-05-04T12:18:24.777104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:47:19.879004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777408+0000","last_change":"2024-05-04T12:18:21.831214+0000","last_active":"2024-05-04T12:18:24.777408+0000","last_peered":"2024-05-04T12:18:24.777408+0000","last_clean":"2024-05-04T12:18:24.777408+0000","last_became_active":"2024-05-04T12:18:21.830771+0000","last_became_peered":"2024-05-04T12:18:21.830771+0000","last_unstale":"2024-05-04T12:18:24.777408+0000","last_undegraded":"2024-05-04T12:18:24.777408+0000","last_fullsized":"2024-05-04T12:18:24.777408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:27:35.940816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296985+0000","last_change":"2024-05-04T12:18:22.852135+0000","last_active":"2024-05-04T12:18:24.296985+0000","last_peered":"2024-05-04T12:18:24.296985+0000","last_clean":"2024-05-04T12:18:24.296985+0000","last_became_active":"2024-05-04T12:18:22.851686+0000","last_became_peered":"2024-05-04T12:18:22.851686+0000","last_unstale":"2024-05-04T12:18:24.296985+0000","last_undegraded":"2024-05-04T12:18:24.296985+0000","last_fullsized":"2024-05-04T12:18:24.296985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:40:13.207623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836318+0000","last_change":"2024-05-04T12:18:21.830621+0000","last_active":"2024-05-04T12:18:23.836318+0000","last_peered":"2024-05-04T12:18:23.836318+0000","last_clean":"2024-05-04T12:18:23.836318+0000","last_became_active":"2024-05-04T12:18:21.829681+0000","last_became_peered":"2024-05-04T12:18:21.829681+0000","last_unstale":"2024-05-04T12:18:23.836318+0000","last_undegraded":"2024-05-04T12:18:23.836318+0000","last_fullsized":"2024-05-04T12:18:23.836318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:04:24.159320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297861+0000","last_change":"2024-05-04T12:18:22.855366+0000","last_active":"2024-05-04T12:18:24.297861+0000","last_peered":"2024-05-04T12:18:24.297861+0000","last_clean":"2024-05-04T12:18:24.297861+0000","last_became_active":"2024-05-04T12:18:22.855103+0000","last_became_peered":"2024-05-04T12:18:22.855103+0000","last_unstale":"2024-05-04T12:18:24.297861+0000","last_undegraded":"2024-05-04T12:18:24.297861+0000","last_fullsized":"2024-05-04T12:18:24.297861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:00:25.310837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778378+0000","last_change":"2024-05-04T12:18:21.832957+0000","last_active":"2024-05-04T12:18:24.778378+0000","last_peered":"2024-05-04T12:18:24.778378+0000","last_clean":"2024-05-04T12:18:24.778378+0000","last_became_active":"2024-05-04T12:18:21.832274+0000","last_became_peered":"2024-05-04T12:18:21.832274+0000","last_unstale":"2024-05-04T12:18:24.778378+0000","last_undegraded":"2024-05-04T12:18:24.778378+0000","last_fullsized":"2024-05-04T12:18:24.778378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298277+0000","last_change":"2024-05-04T12:18:22.861628+0000","last_active":"2024-05-04T12:18:24.298277+0000","last_peered":"2024-05-04T12:18:24.298277+0000","last_clean":"2024-05-04T12:18:24.298277+0000","last_became_active":"2024-05-04T12:18:22.861348+0000","last_became_peered":"2024-05-04T12:18:22.861348+0000","last_unstale":"2024-05-04T12:18:24.298277+0000","last_undegraded":"2024-05-04T12:18:24.298277+0000","last_fullsized":"2024-05-04T12:18:24.298277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:21:56.962539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.832228+0000","last_change":"2024-05-04T12:18:21.832502+0000","last_active":"2024-05-04T12:18:23.832228+0000","last_peered":"2024-05-04T12:18:23.832228+0000","last_clean":"2024-05-04T12:18:23.832228+0000","last_became_active":"2024-05-04T12:18:21.832085+0000","last_became_peered":"2024-05-04T12:18:21.832085+0000","last_unstale":"2024-05-04T12:18:23.832228+0000","last_undegraded":"2024-05-04T12:18:23.832228+0000","last_fullsized":"2024-05-04T12:18:23.832228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:39.485391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836949+0000","last_change":"2024-05-04T12:18:22.870045+0000","last_active":"2024-05-04T12:18:23.836949+0000","last_peered":"2024-05-04T12:18:23.836949+0000","last_clean":"2024-05-04T12:18:23.836949+0000","last_became_active":"2024-05-04T12:18:22.869646+0000","last_became_peered":"2024-05-04T12:18:22.869646+0000","last_unstale":"2024-05-04T12:18:23.836949+0000","last_undegraded":"2024-05-04T12:18:23.836949+0000","last_fullsized":"2024-05-04T12:18:23.836949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:02:37.747970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777797+0000","last_change":"2024-05-04T12:18:21.839604+0000","last_active":"2024-05-04T12:18:24.777797+0000","last_peered":"2024-05-04T12:18:24.777797+0000","last_clean":"2024-05-04T12:18:24.777797+0000","last_became_active":"2024-05-04T12:18:21.839304+0000","last_became_peered":"2024-05-04T12:18:21.839304+0000","last_unstale":"2024-05-04T12:18:24.777797+0000","last_undegraded":"2024-05-04T12:18:24.777797+0000","last_fullsized":"2024-05-04T12:18:24.777797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297752+0000","last_change":"2024-05-04T12:18:22.854672+0000","last_active":"2024-05-04T12:18:24.297752+0000","last_peered":"2024-05-04T12:18:24.297752+0000","last_clean":"2024-05-04T12:18:24.297752+0000","last_became_active":"2024-05-04T12:18:22.852798+0000","last_became_peered":"2024-05-04T12:18:22.852798+0000","last_unstale":"2024-05-04T12:18:24.297752+0000","last_undegraded":"2024-05-04T12:18:24.297752+0000","last_fullsized":"2024-05-04T12:18:24.297752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:16:56.544684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296796+0000","last_change":"2024-05-04T12:18:12.644577+0000","last_active":"2024-05-04T12:18:24.296796+0000","last_peered":"2024-05-04T12:18:24.296796+0000","last_clean":"2024-05-04T12:18:24.296796+0000","last_became_active":"2024-05-04T12:18:12.640041+0000","last_became_peered":"2024-05-04T12:18:12.640041+0000","last_unstale":"2024-05-04T12:18:24.296796+0000","last_undegraded":"2024-05-04T12:18:24.296796+0000","last_fullsized":"2024-05-04T12:18:24.296796+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_clean_scrub_stamp":"2024-05-04T12:18:11.260157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:15:02.089387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836180+0000","last_change":"2024-05-04T12:18:21.837361+0000","last_active":"2024-05-04T12:18:23.836180+0000","last_peered":"2024-05-04T12:18:23.836180+0000","last_clean":"2024-05-04T12:18:23.836180+0000","last_became_active":"2024-05-04T12:18:21.836437+0000","last_became_peered":"2024-05-04T12:18:21.836437+0000","last_unstale":"2024-05-04T12:18:23.836180+0000","last_undegraded":"2024-05-04T12:18:23.836180+0000","last_fullsized":"2024-05-04T12:18:23.836180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:48:12.147734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779602+0000","last_change":"2024-05-04T12:18:22.868591+0000","last_active":"2024-05-04T12:18:24.779602+0000","last_peered":"2024-05-04T12:18:24.779602+0000","last_clean":"2024-05-04T12:18:24.779602+0000","last_became_active":"2024-05-04T12:18:22.868158+0000","last_became_peered":"2024-05-04T12:18:22.868158+0000","last_unstale":"2024-05-04T12:18:24.779602+0000","last_undegraded":"2024-05-04T12:18:24.779602+0000","last_fullsized":"2024-05-04T12:18:24.779602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:05:46.918761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296711+0000","last_change":"2024-05-04T12:18:21.857252+0000","last_active":"2024-05-04T12:18:24.296711+0000","last_peered":"2024-05-04T12:18:24.296711+0000","last_clean":"2024-05-04T12:18:24.296711+0000","last_became_active":"2024-05-04T12:18:21.855446+0000","last_became_peered":"2024-05-04T12:18:21.855446+0000","last_unstale":"2024-05-04T12:18:24.296711+0000","last_undegraded":"2024-05-04T12:18:24.296711+0000","last_fullsized":"2024-05-04T12:18:24.296711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.246868+0000","last_change":"2024-05-04T12:18:22.862302+0000","last_active":"2024-05-04T12:18:24.246868+0000","last_peered":"2024-05-04T12:18:24.246868+0000","last_clean":"2024-05-04T12:18:24.246868+0000","last_became_active":"2024-05-04T12:18:22.859627+0000","last_became_peered":"2024-05-04T12:18:22.859627+0000","last_unstale":"2024-05-04T12:18:24.246868+0000","last_undegraded":"2024-05-04T12:18:24.246868+0000","last_fullsized":"2024-05-04T12:18:24.246868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:16:07.430915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778081+0000","last_change":"2024-05-04T12:18:21.832787+0000","last_active":"2024-05-04T12:18:24.778081+0000","last_peered":"2024-05-04T12:18:24.778081+0000","last_clean":"2024-05-04T12:18:24.778081+0000","last_became_active":"2024-05-04T12:18:21.832247+0000","last_became_peered":"2024-05-04T12:18:21.832247+0000","last_unstale":"2024-05-04T12:18:24.778081+0000","last_undegraded":"2024-05-04T12:18:24.778081+0000","last_fullsized":"2024-05-04T12:18:24.778081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:48:12.147734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831403+0000","last_change":"2024-05-04T12:18:22.863972+0000","last_active":"2024-05-04T12:18:23.831403+0000","last_peered":"2024-05-04T12:18:23.831403+0000","last_clean":"2024-05-04T12:18:23.831403+0000","last_became_active":"2024-05-04T12:18:22.863122+0000","last_became_peered":"2024-05-04T12:18:22.863122+0000","last_unstale":"2024-05-04T12:18:23.831403+0000","last_undegraded":"2024-05-04T12:18:23.831403+0000","last_fullsized":"2024-05-04T12:18:23.831403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:47:19.879004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836266+0000","last_change":"2024-05-04T12:18:21.835740+0000","last_active":"2024-05-04T12:18:23.836266+0000","last_peered":"2024-05-04T12:18:23.836266+0000","last_clean":"2024-05-04T12:18:23.836266+0000","last_became_active":"2024-05-04T12:18:21.834835+0000","last_became_peered":"2024-05-04T12:18:21.834835+0000","last_unstale":"2024-05-04T12:18:23.836266+0000","last_undegraded":"2024-05-04T12:18:23.836266+0000","last_fullsized":"2024-05-04T12:18:23.836266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835634+0000","last_change":"2024-05-04T12:18:22.856686+0000","last_active":"2024-05-04T12:18:23.835634+0000","last_peered":"2024-05-04T12:18:23.835634+0000","last_clean":"2024-05-04T12:18:23.835634+0000","last_became_active":"2024-05-04T12:18:22.855883+0000","last_became_peered":"2024-05-04T12:18:22.855883+0000","last_unstale":"2024-05-04T12:18:23.835634+0000","last_undegraded":"2024-05-04T12:18:23.835634+0000","last_fullsized":"2024-05-04T12:18:23.835634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:19.851400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831728+0000","last_change":"2024-05-04T12:18:21.831578+0000","last_active":"2024-05-04T12:18:23.831728+0000","last_peered":"2024-05-04T12:18:23.831728+0000","last_clean":"2024-05-04T12:18:23.831728+0000","last_became_active":"2024-05-04T12:18:21.830473+0000","last_became_peered":"2024-05-04T12:18:21.830473+0000","last_unstale":"2024-05-04T12:18:23.831728+0000","last_undegraded":"2024-05-04T12:18:23.831728+0000","last_fullsized":"2024-05-04T12:18:23.831728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:19.610922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.837084+0000","last_change":"2024-05-04T12:18:22.878341+0000","last_active":"2024-05-04T12:18:23.837084+0000","last_peered":"2024-05-04T12:18:23.837084+0000","last_clean":"2024-05-04T12:18:23.837084+0000","last_became_active":"2024-05-04T12:18:22.878015+0000","last_became_peered":"2024-05-04T12:18:22.878015+0000","last_unstale":"2024-05-04T12:18:23.837084+0000","last_undegraded":"2024-05-04T12:18:23.837084+0000","last_fullsized":"2024-05-04T12:18:23.837084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:00:25.310837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.298025+0000","last_change":"2024-05-04T12:18:21.857646+0000","last_active":"2024-05-04T12:18:24.298025+0000","last_peered":"2024-05-04T12:18:24.298025+0000","last_clean":"2024-05-04T12:18:24.298025+0000","last_became_active":"2024-05-04T12:18:21.856638+0000","last_became_peered":"2024-05-04T12:18:21.856638+0000","last_unstale":"2024-05-04T12:18:24.298025+0000","last_undegraded":"2024-05-04T12:18:24.298025+0000","last_fullsized":"2024-05-04T12:18:24.298025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:09:02.023381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831578+0000","last_change":"2024-05-04T12:18:22.862113+0000","last_active":"2024-05-04T12:18:23.831578+0000","last_peered":"2024-05-04T12:18:23.831578+0000","last_clean":"2024-05-04T12:18:23.831578+0000","last_became_active":"2024-05-04T12:18:22.861682+0000","last_became_peered":"2024-05-04T12:18:22.861682+0000","last_unstale":"2024-05-04T12:18:23.831578+0000","last_undegraded":"2024-05-04T12:18:23.831578+0000","last_fullsized":"2024-05-04T12:18:23.831578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.636826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.775560+0000","last_change":"2024-05-04T12:18:21.846869+0000","last_active":"2024-05-04T12:18:24.775560+0000","last_peered":"2024-05-04T12:18:24.775560+0000","last_clean":"2024-05-04T12:18:24.775560+0000","last_became_active":"2024-05-04T12:18:21.846514+0000","last_became_peered":"2024-05-04T12:18:21.846514+0000","last_unstale":"2024-05-04T12:18:24.775560+0000","last_undegraded":"2024-05-04T12:18:24.775560+0000","last_fullsized":"2024-05-04T12:18:24.775560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779211+0000","last_change":"2024-05-04T12:18:22.879784+0000","last_active":"2024-05-04T12:18:24.779211+0000","last_peered":"2024-05-04T12:18:24.779211+0000","last_clean":"2024-05-04T12:18:24.779211+0000","last_became_active":"2024-05-04T12:18:22.877733+0000","last_became_peered":"2024-05-04T12:18:22.877733+0000","last_unstale":"2024-05-04T12:18:24.779211+0000","last_undegraded":"2024-05-04T12:18:24.779211+0000","last_fullsized":"2024-05-04T12:18:24.779211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:10:47.797595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.777681+0000","last_change":"2024-05-04T12:18:21.854368+0000","last_active":"2024-05-04T12:18:24.777681+0000","last_peered":"2024-05-04T12:18:24.777681+0000","last_clean":"2024-05-04T12:18:24.777681+0000","last_became_active":"2024-05-04T12:18:21.852721+0000","last_became_peered":"2024-05-04T12:18:21.852721+0000","last_unstale":"2024-05-04T12:18:24.777681+0000","last_undegraded":"2024-05-04T12:18:24.777681+0000","last_fullsized":"2024-05-04T12:18:24.777681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247454+0000","last_change":"2024-05-04T12:18:22.853292+0000","last_active":"2024-05-04T12:18:24.247454+0000","last_peered":"2024-05-04T12:18:24.247454+0000","last_clean":"2024-05-04T12:18:24.247454+0000","last_became_active":"2024-05-04T12:18:22.852401+0000","last_became_peered":"2024-05-04T12:18:22.852401+0000","last_unstale":"2024-05-04T12:18:24.247454+0000","last_undegraded":"2024-05-04T12:18:24.247454+0000","last_fullsized":"2024-05-04T12:18:24.247454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.296652+0000","last_change":"2024-05-04T12:18:21.858135+0000","last_active":"2024-05-04T12:18:24.296652+0000","last_peered":"2024-05-04T12:18:24.296652+0000","last_clean":"2024-05-04T12:18:24.296652+0000","last_became_active":"2024-05-04T12:18:21.856430+0000","last_became_peered":"2024-05-04T12:18:21.856430+0000","last_unstale":"2024-05-04T12:18:24.296652+0000","last_undegraded":"2024-05-04T12:18:24.296652+0000","last_fullsized":"2024-05-04T12:18:24.296652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:22.478652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.835773+0000","last_change":"2024-05-04T12:18:22.852562+0000","last_active":"2024-05-04T12:18:23.835773+0000","last_peered":"2024-05-04T12:18:23.835773+0000","last_clean":"2024-05-04T12:18:23.835773+0000","last_became_active":"2024-05-04T12:18:22.851980+0000","last_became_peered":"2024-05-04T12:18:22.851980+0000","last_unstale":"2024-05-04T12:18:23.835773+0000","last_undegraded":"2024-05-04T12:18:23.835773+0000","last_fullsized":"2024-05-04T12:18:23.835773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:14:23.487598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831190+0000","last_change":"2024-05-04T12:18:21.837963+0000","last_active":"2024-05-04T12:18:23.831190+0000","last_peered":"2024-05-04T12:18:23.831190+0000","last_clean":"2024-05-04T12:18:23.831190+0000","last_became_active":"2024-05-04T12:18:21.837563+0000","last_became_peered":"2024-05-04T12:18:21.837563+0000","last_unstale":"2024-05-04T12:18:23.831190+0000","last_undegraded":"2024-05-04T12:18:23.831190+0000","last_fullsized":"2024-05-04T12:18:23.831190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:58.968143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.778827+0000","last_change":"2024-05-04T12:18:22.874277+0000","last_active":"2024-05-04T12:18:24.778827+0000","last_peered":"2024-05-04T12:18:24.778827+0000","last_clean":"2024-05-04T12:18:24.778827+0000","last_became_active":"2024-05-04T12:18:22.869298+0000","last_became_peered":"2024-05-04T12:18:22.869298+0000","last_unstale":"2024-05-04T12:18:24.778827+0000","last_undegraded":"2024-05-04T12:18:24.778827+0000","last_fullsized":"2024-05-04T12:18:24.778827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:35:22.275376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.831728+0000","last_change":"2024-05-04T12:18:21.831177+0000","last_active":"2024-05-04T12:18:23.831728+0000","last_peered":"2024-05-04T12:18:23.831728+0000","last_clean":"2024-05-04T12:18:23.831728+0000","last_became_active":"2024-05-04T12:18:21.830570+0000","last_became_peered":"2024-05-04T12:18:21.830570+0000","last_unstale":"2024-05-04T12:18:23.831728+0000","last_undegraded":"2024-05-04T12:18:23.831728+0000","last_fullsized":"2024-05-04T12:18:23.831728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:10:18.842454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779529+0000","last_change":"2024-05-04T12:18:22.880190+0000","last_active":"2024-05-04T12:18:24.779529+0000","last_peered":"2024-05-04T12:18:24.779529+0000","last_clean":"2024-05-04T12:18:24.779529+0000","last_became_active":"2024-05-04T12:18:22.879801+0000","last_became_peered":"2024-05-04T12:18:22.879801+0000","last_unstale":"2024-05-04T12:18:24.779529+0000","last_undegraded":"2024-05-04T12:18:24.779529+0000","last_fullsized":"2024-05-04T12:18:24.779529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:40:13.207623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297558+0000","last_change":"2024-05-04T12:18:21.860830+0000","last_active":"2024-05-04T12:18:24.297558+0000","last_peered":"2024-05-04T12:18:24.297558+0000","last_clean":"2024-05-04T12:18:24.297558+0000","last_became_active":"2024-05-04T12:18:21.860156+0000","last_became_peered":"2024-05-04T12:18:21.860156+0000","last_unstale":"2024-05-04T12:18:24.297558+0000","last_undegraded":"2024-05-04T12:18:24.297558+0000","last_fullsized":"2024-05-04T12:18:24.297558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:31:39.485391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.776930+0000","last_change":"2024-05-04T12:18:22.860205+0000","last_active":"2024-05-04T12:18:24.776930+0000","last_peered":"2024-05-04T12:18:24.776930+0000","last_clean":"2024-05-04T12:18:24.776930+0000","last_became_active":"2024-05-04T12:18:22.859825+0000","last_became_peered":"2024-05-04T12:18:22.859825+0000","last_unstale":"2024-05-04T12:18:24.776930+0000","last_undegraded":"2024-05-04T12:18:24.776930+0000","last_fullsized":"2024-05-04T12:18:24.776930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:18:08.570317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.837078+0000","last_change":"2024-05-04T12:18:22.844339+0000","last_active":"2024-05-04T12:18:23.837078+0000","last_peered":"2024-05-04T12:18:23.837078+0000","last_clean":"2024-05-04T12:18:23.837078+0000","last_became_active":"2024-05-04T12:18:22.843807+0000","last_became_peered":"2024-05-04T12:18:22.843807+0000","last_unstale":"2024-05-04T12:18:23.837078+0000","last_undegraded":"2024-05-04T12:18:23.837078+0000","last_fullsized":"2024-05-04T12:18:23.837078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:16:56.544684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297584+0000","last_change":"2024-05-04T12:18:21.860670+0000","last_active":"2024-05-04T12:18:24.297584+0000","last_peered":"2024-05-04T12:18:24.297584+0000","last_clean":"2024-05-04T12:18:24.297584+0000","last_became_active":"2024-05-04T12:18:21.859826+0000","last_became_peered":"2024-05-04T12:18:21.859826+0000","last_unstale":"2024-05-04T12:18:24.297584+0000","last_undegraded":"2024-05-04T12:18:24.297584+0000","last_fullsized":"2024-05-04T12:18:24.297584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:50:36.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:23.836737+0000","last_change":"2024-05-04T12:18:22.846218+0000","last_active":"2024-05-04T12:18:23.836737+0000","last_peered":"2024-05-04T12:18:23.836737+0000","last_clean":"2024-05-04T12:18:23.836737+0000","last_became_active":"2024-05-04T12:18:22.845967+0000","last_became_peered":"2024-05-04T12:18:22.845967+0000","last_unstale":"2024-05-04T12:18:23.836737+0000","last_undegraded":"2024-05-04T12:18:23.836737+0000","last_fullsized":"2024-05-04T12:18:23.836737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:05:46.918761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.297709+0000","last_change":"2024-05-04T12:18:21.860069+0000","last_active":"2024-05-04T12:18:24.297709+0000","last_peered":"2024-05-04T12:18:24.297709+0000","last_clean":"2024-05-04T12:18:24.297709+0000","last_became_active":"2024-05-04T12:18:21.859583+0000","last_became_peered":"2024-05-04T12:18:21.859583+0000","last_unstale":"2024-05-04T12:18:24.297709+0000","last_undegraded":"2024-05-04T12:18:24.297709+0000","last_fullsized":"2024-05-04T12:18:24.297709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:03:35.131163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247407+0000","last_change":"2024-05-04T12:18:22.856277+0000","last_active":"2024-05-04T12:18:24.247407+0000","last_peered":"2024-05-04T12:18:24.247407+0000","last_clean":"2024-05-04T12:18:24.247407+0000","last_became_active":"2024-05-04T12:18:22.855958+0000","last_became_peered":"2024-05-04T12:18:22.855958+0000","last_unstale":"2024-05-04T12:18:24.247407+0000","last_undegraded":"2024-05-04T12:18:24.247407+0000","last_fullsized":"2024-05-04T12:18:24.247407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:35:04.636826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779181+0000","last_change":"2024-05-04T12:18:21.835661+0000","last_active":"2024-05-04T12:18:24.779181+0000","last_peered":"2024-05-04T12:18:24.779181+0000","last_clean":"2024-05-04T12:18:24.779181+0000","last_became_active":"2024-05-04T12:18:21.835299+0000","last_became_peered":"2024-05-04T12:18:21.835299+0000","last_unstale":"2024-05-04T12:18:24.779181+0000","last_undegraded":"2024-05-04T12:18:24.779181+0000","last_fullsized":"2024-05-04T12:18:24.779181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:09:02.023381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.779181+0000","last_change":"2024-05-04T12:18:22.878092+0000","last_active":"2024-05-04T12:18:24.779181+0000","last_peered":"2024-05-04T12:18:24.779181+0000","last_clean":"2024-05-04T12:18:24.779181+0000","last_became_active":"2024-05-04T12:18:22.875650+0000","last_became_peered":"2024-05-04T12:18:22.875650+0000","last_unstale":"2024-05-04T12:18:24.779181+0000","last_undegraded":"2024-05-04T12:18:24.779181+0000","last_fullsized":"2024-05-04T12:18:24.779181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:21.803391+0000","last_clean_scrub_stamp":"2024-05-04T12:18:21.803391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T18:04:13.180987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-04T12:18:24.247663+0000","last_change":"2024-05-04T12:18:21.847981+0000","last_active":"2024-05-04T12:18:24.247663+0000","last_peered":"2024-05-04T12:18:24.247663+0000","last_clean":"2024-05-04T12:18:24.247663+0000","last_became_active":"2024-05-04T12:18:21.847556+0000","last_became_peered":"2024-05-04T12:18:21.847556+0000","last_unstale":"2024-05-04T12:18:24.247663+0000","last_undegraded":"2024-05-04T12:18:24.247663+0000","last_fullsized":"2024-05-04T12:18:24.247663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:20.794446+0000","last_clean_scrub_stamp":"2024-05-04T12:18:20.794446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:07:48.655726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T12:18:29.985 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-04T12:18:29.986 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-04T12:18:29.986 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-04T12:18:29.986 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-04T12:18:30.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.142+0000 7fdde92e6640 1 Processor -- start 2024-05-04T12:18:30.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.142+0000 7fdde92e6640 1 -- start start 2024-05-04T12:18:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.142+0000 7fdde92e6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdde4070930 0x7fdde41601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.142+0000 7fdde92e6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdde415c870 0x7fdde415cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.142+0000 7fdde92e6640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdde406dd90 con 0x7fdde4070d10 2024-05-04T12:18:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.142+0000 7fdde92e6640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fdde406e000 con 0x7fdde4070930 2024-05-04T12:18:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.142+0000 7fdde92e6640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fdde406e2e0 con 0x7fdde415c870 2024-05-04T12:18:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.142+0000 7fdde2ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdde4070930 0x7fdde41601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.142+0000 7fdde27fc640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fdde4070d10 0x7fdde4160700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51584/0 (socket says 172.21.15.52:51584) 2024-05-04T12:18:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.142+0000 7fdde27fc640 1 -- 172.21.15.52:0/1003948252 learned_addr learned my addr 172.21.15.52:0/1003948252 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.146+0000 7fdde37fe640 1 --2- 172.21.15.52:0/1003948252 >> v2:172.21.15.111:3300/0 conn(0x7fdde415c870 0x7fdde415cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.146+0000 7fdde2ffd640 1 -- 172.21.15.52:0/1003948252 >> v2:172.21.15.111:3300/0 conn(0x7fdde415c870 msgr2=0x7fdde415cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.146+0000 7fdde2ffd640 1 --2- 172.21.15.52:0/1003948252 >> v2:172.21.15.111:3300/0 conn(0x7fdde415c870 0x7fdde415cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.146+0000 7fdde2ffd640 1 -- 172.21.15.52:0/1003948252 >> v1:172.21.15.52:6789/0 conn(0x7fdde4070d10 legacy=0x7fdde4160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.146+0000 7fdde2ffd640 1 -- 172.21.15.52:0/1003948252 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde406e5c0 con 0x7fdde4070930 2024-05-04T12:18:30.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.146+0000 7fdde1ffb640 1 -- 172.21.15.52:0/1003948252 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4076657539 0 0) 0x7fdde406dd90 con 0x7fdde4070d10 2024-05-04T12:18:30.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.146+0000 7fdde2ffd640 1 --2- 172.21.15.52:0/1003948252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdde4070930 0x7fdde41601c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdde40711d0 tx=0x7fddd401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3b63bbf736c73a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.146+0000 7fdde1ffb640 1 -- 172.21.15.52:0/1003948252 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd40b1070 con 0x7fdde4070930 2024-05-04T12:18:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.146+0000 7fdde1ffb640 1 -- 172.21.15.52:0/1003948252 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd40ac600 con 0x7fdde4070930 2024-05-04T12:18:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.146+0000 7fdde1ffb640 1 -- 172.21.15.52:0/1003948252 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd40ac9d0 con 0x7fdde4070930 2024-05-04T12:18:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 -- 172.21.15.52:0/1003948252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdde4070930 msgr2=0x7fdde41601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 --2- 172.21.15.52:0/1003948252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdde4070930 0x7fdde41601c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdde40711d0 tx=0x7fddd401c480 comp rx=0 tx=0).stop 2024-05-04T12:18:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 -- 172.21.15.52:0/1003948252 shutdown_connections 2024-05-04T12:18:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 --2- 172.21.15.52:0/1003948252 >> v2:172.21.15.111:3300/0 conn(0x7fdde415c870 0x7fdde415cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 --2- 172.21.15.52:0/1003948252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdde4070930 0x7fdde41601c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 -- 172.21.15.52:0/1003948252 >> 172.21.15.52:0/1003948252 conn(0x7fdde4067070 msgr2=0x7fdde4067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 -- 172.21.15.52:0/1003948252 shutdown_connections 2024-05-04T12:18:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 -- 172.21.15.52:0/1003948252 wait complete. 2024-05-04T12:18:30.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 Processor -- start 2024-05-04T12:18:30.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 -- start start 2024-05-04T12:18:30.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde4070930 0x7fdde41519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde4070d10 0x7fdde413db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdde415c870 0x7fdde41556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4168a20 con 0x7fdde415c870 2024-05-04T12:18:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41688a0 con 0x7fdde4070930 2024-05-04T12:18:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fdde4168ba0 con 0x7fdde4070d10 2024-05-04T12:18:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdde415c870 0x7fdde41556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdde415c870 0x7fdde41556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53656/0 (socket says 172.21.15.52:53656) 2024-05-04T12:18:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde37fe640 1 -- 172.21.15.52:0/3745422030 learned_addr learned my addr 172.21.15.52:0/3745422030 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde2ffd640 1 --2- 172.21.15.52:0/3745422030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde4070930 0x7fdde41519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde27fc640 1 --2- 172.21.15.52:0/3745422030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde4070d10 0x7fdde413db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde37fe640 1 -- 172.21.15.52:0/3745422030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde4070d10 msgr2=0x7fdde413db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde37fe640 1 --2- 172.21.15.52:0/3745422030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde4070d10 0x7fdde413db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde37fe640 1 -- 172.21.15.52:0/3745422030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde4070930 msgr2=0x7fdde41519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde37fe640 1 --2- 172.21.15.52:0/3745422030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde4070930 0x7fdde41519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde37fe640 1 -- 172.21.15.52:0/3745422030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde4137420 con 0x7fdde415c870 2024-05-04T12:18:30.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde37fe640 1 --2- 172.21.15.52:0/3745422030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdde415c870 0x7fdde41556a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdddc007e40 tx=0x7fdddc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:30.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fddcb7fe640 1 -- 172.21.15.52:0/3745422030 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc014620 con 0x7fdde415c870 2024-05-04T12:18:30.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fddcb7fe640 1 -- 172.21.15.52:0/3745422030 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdddc019070 con 0x7fdde415c870 2024-05-04T12:18:30.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fddcb7fe640 1 -- 172.21.15.52:0/3745422030 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc014a20 con 0x7fdde415c870 2024-05-04T12:18:30.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 -- 172.21.15.52:0/3745422030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde4151ef0 con 0x7fdde415c870 2024-05-04T12:18:30.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 -- 172.21.15.52:0/3745422030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde41379a0 con 0x7fdde415c870 2024-05-04T12:18:30.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.150+0000 7fdde92e6640 1 -- 172.21.15.52:0/3745422030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde4062e50 con 0x7fdde415c870 2024-05-04T12:18:30.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.154+0000 7fddcb7fe640 1 -- 172.21.15.52:0/3745422030 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdddc014bc0 con 0x7fdde415c870 2024-05-04T12:18:30.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.154+0000 7fddcb7fe640 1 --2- 172.21.15.52:0/3745422030 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fddd00678c0 0x7fddd0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.154+0000 7fddcb7fe640 1 -- 172.21.15.52:0/3745422030 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdddc0864d0 con 0x7fdde415c870 2024-05-04T12:18:30.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.154+0000 7fdde2ffd640 1 --2- 172.21.15.52:0/3745422030 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fddd00678c0 0x7fddd0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.154+0000 7fdde2ffd640 1 --2- 172.21.15.52:0/3745422030 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fddd00678c0 0x7fddd0069d80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fddd40ae000 tx=0x7fddd40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:30.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.158+0000 7fddcb7fe640 1 -- 172.21.15.52:0/3745422030 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdddc0570a0 con 0x7fdde415c870 2024-05-04T12:18:30.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.346+0000 7fdde92e6640 1 -- 172.21.15.52:0/3745422030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fdde4075320 con 0x7fdde415c870 2024-05-04T12:18:30.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.350+0000 7fddcb7fe640 1 -- 172.21.15.52:0/3745422030 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fdddc0569a0 con 0x7fdde415c870 2024-05-04T12:18:30.351 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:30.351 INFO:teuthology.orchestra.run.smithi052.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-04T12:18:30.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.350+0000 7fdde92e6640 1 -- 172.21.15.52:0/3745422030 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fddd00678c0 msgr2=0x7fddd0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.350+0000 7fdde92e6640 1 --2- 172.21.15.52:0/3745422030 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fddd00678c0 0x7fddd0069d80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fddd40ae000 tx=0x7fddd40a7000 comp rx=0 tx=0).stop 2024-05-04T12:18:30.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.350+0000 7fdde92e6640 1 -- 172.21.15.52:0/3745422030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdde415c870 msgr2=0x7fdde41556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.350+0000 7fdde92e6640 1 --2- 172.21.15.52:0/3745422030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdde415c870 0x7fdde41556a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdddc007e40 tx=0x7fdddc00e4b0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.350+0000 7fdde92e6640 1 -- 172.21.15.52:0/3745422030 shutdown_connections 2024-05-04T12:18:30.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.350+0000 7fdde92e6640 1 --2- 172.21.15.52:0/3745422030 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fddd00678c0 0x7fddd0069d80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.350+0000 7fdde92e6640 1 --2- 172.21.15.52:0/3745422030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdde415c870 0x7fdde41556a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.350+0000 7fdde92e6640 1 --2- 172.21.15.52:0/3745422030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde4070d10 0x7fdde413db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.350+0000 7fdde92e6640 1 --2- 172.21.15.52:0/3745422030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde4070930 0x7fdde41519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.350+0000 7fdde92e6640 1 -- 172.21.15.52:0/3745422030 >> 172.21.15.52:0/3745422030 conn(0x7fdde4067070 msgr2=0x7fdde415ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:30.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.350+0000 7fdde92e6640 1 -- 172.21.15.52:0/3745422030 shutdown_connections 2024-05-04T12:18:30.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.350+0000 7fdde92e6640 1 -- 172.21.15.52:0/3745422030 wait complete. 2024-05-04T12:18:30.368 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-04T12:18:30.368 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:30.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd87d1640 1 Processor -- start 2024-05-04T12:18:30.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd87d1640 1 -- start start 2024-05-04T12:18:30.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd87d1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fcdd00cd5c0 0x7fcdd006bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd87d1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcdd015c700 0x7fcdd015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd87d1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd0141c80 con 0x7fcdd01417d0 2024-05-04T12:18:30.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd87d1640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd013f490 con 0x7fcdd00cd5c0 2024-05-04T12:18:30.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd87d1640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd015d050 con 0x7fcdd015c700 2024-05-04T12:18:30.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd5d6c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fcdd01417d0 0x7fcdd01603b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51596/0 (socket says 172.21.15.52:51596) 2024-05-04T12:18:30.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd656d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fcdd00cd5c0 0x7fcdd006bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd5d6c640 1 -- 172.21.15.52:0/2574043491 learned_addr learned my addr 172.21.15.52:0/2574043491 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:30.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd6d6e640 1 --2- 172.21.15.52:0/2574043491 >> v2:172.21.15.111:3300/0 conn(0x7fcdd015c700 0x7fcdd015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd656d640 1 -- 172.21.15.52:0/2574043491 >> v2:172.21.15.111:3300/0 conn(0x7fcdd015c700 msgr2=0x7fcdd015cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:30.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd656d640 1 --2- 172.21.15.52:0/2574043491 >> v2:172.21.15.111:3300/0 conn(0x7fcdd015c700 0x7fcdd015cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd656d640 1 -- 172.21.15.52:0/2574043491 >> v1:172.21.15.52:6789/0 conn(0x7fcdd01417d0 legacy=0x7fcdd01603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd556b640 1 -- 172.21.15.52:0/2574043491 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2332245404 0 0) 0x7fcdc4001120 con 0x7fcdd01417d0 2024-05-04T12:18:30.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.486+0000 7fcdd656d640 1 -- 172.21.15.52:0/2574043491 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd0141f00 con 0x7fcdd00cd5c0 2024-05-04T12:18:30.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.490+0000 7fcdd656d640 1 --2- 172.21.15.52:0/2574043491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd00cd5c0 0x7fcdd006bb70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcdd015fc30 tx=0x7fcdc001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e5369973351eb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:30.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.490+0000 7fcdd556b640 1 -- 172.21.15.52:0/2574043491 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc00b1070 con 0x7fcdd00cd5c0 2024-05-04T12:18:30.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.490+0000 7fcdd556b640 1 -- 172.21.15.52:0/2574043491 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc00ac5f0 con 0x7fcdd00cd5c0 2024-05-04T12:18:30.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.490+0000 7fcdd556b640 1 -- 172.21.15.52:0/2574043491 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc00ac9c0 con 0x7fcdd00cd5c0 2024-05-04T12:18:30.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.490+0000 7fcdd87d1640 1 -- 172.21.15.52:0/2574043491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd00cd5c0 msgr2=0x7fcdd006bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.490+0000 7fcdd87d1640 1 --2- 172.21.15.52:0/2574043491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd00cd5c0 0x7fcdd006bb70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcdd015fc30 tx=0x7fcdc001c480 comp rx=0 tx=0).stop 2024-05-04T12:18:30.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.490+0000 7fcdd87d1640 1 -- 172.21.15.52:0/2574043491 shutdown_connections 2024-05-04T12:18:30.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.490+0000 7fcdd87d1640 1 --2- 172.21.15.52:0/2574043491 >> v2:172.21.15.111:3300/0 conn(0x7fcdd015c700 0x7fcdd015cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.490+0000 7fcdd87d1640 1 --2- 172.21.15.52:0/2574043491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd00cd5c0 0x7fcdd006bb70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.490+0000 7fcdd87d1640 1 -- 172.21.15.52:0/2574043491 >> 172.21.15.52:0/2574043491 conn(0x7fcdd00b2380 msgr2=0x7fcdd00b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:30.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.490+0000 7fcdd87d1640 1 -- 172.21.15.52:0/2574043491 shutdown_connections 2024-05-04T12:18:30.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.490+0000 7fcdd87d1640 1 -- 172.21.15.52:0/2574043491 wait complete. 2024-05-04T12:18:30.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd87d1640 1 Processor -- start 2024-05-04T12:18:30.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd87d1640 1 -- start start 2024-05-04T12:18:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd87d1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdd00cd5c0 0x7fcdd013b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd87d1640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcdd01417d0 0x7fcdd015eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd87d1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd015c700 0x7fcdd01390f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd87d1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0160c40 con 0x7fcdd015c700 2024-05-04T12:18:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd87d1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0160ac0 con 0x7fcdd00cd5c0 2024-05-04T12:18:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd87d1640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd0073360 con 0x7fcdd01417d0 2024-05-04T12:18:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd6d6e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd015c700 0x7fcdd01390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd6d6e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd015c700 0x7fcdd01390f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53676/0 (socket says 172.21.15.52:53676) 2024-05-04T12:18:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd6d6e640 1 -- 172.21.15.52:0/1728788536 learned_addr learned my addr 172.21.15.52:0/1728788536 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd656d640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdd00cd5c0 0x7fcdd013b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd6d6e640 1 -- 172.21.15.52:0/1728788536 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcdd01417d0 msgr2=0x7fcdd015eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd5d6c640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcdd01417d0 0x7fcdd015eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd6d6e640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcdd01417d0 0x7fcdd015eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd6d6e640 1 -- 172.21.15.52:0/1728788536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdd00cd5c0 msgr2=0x7fcdd013b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd6d6e640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdd00cd5c0 0x7fcdd013b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd6d6e640 1 -- 172.21.15.52:0/1728788536 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd015de40 con 0x7fcdd015c700 2024-05-04T12:18:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd656d640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdd00cd5c0 0x7fcdd013b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd5d6c640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcdd01417d0 0x7fcdd015eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd6d6e640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd015c700 0x7fcdd01390f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009d30 tx=0x7fcdc800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdceffd640 1 -- 172.21.15.52:0/1728788536 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8018020 con 0x7fcdd015c700 2024-05-04T12:18:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdceffd640 1 -- 172.21.15.52:0/1728788536 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc8014480 con 0x7fcdd015c700 2024-05-04T12:18:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd87d1640 1 -- 172.21.15.52:0/1728788536 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd015e120 con 0x7fcdd015c700 2024-05-04T12:18:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdceffd640 1 -- 172.21.15.52:0/1728788536 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8014870 con 0x7fcdd015c700 2024-05-04T12:18:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd87d1640 1 -- 172.21.15.52:0/1728788536 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd015e6e0 con 0x7fcdd015c700 2024-05-04T12:18:30.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdd87d1640 1 -- 172.21.15.52:0/1728788536 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd0062e50 con 0x7fcdd015c700 2024-05-04T12:18:30.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdceffd640 1 -- 172.21.15.52:0/1728788536 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcdc801b020 con 0x7fcdd015c700 2024-05-04T12:18:30.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.494+0000 7fcdceffd640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcdbc0678c0 0x7fcdbc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.498+0000 7fcdceffd640 1 -- 172.21.15.52:0/1728788536 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcdc8086660 con 0x7fcdd015c700 2024-05-04T12:18:30.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.498+0000 7fcdd656d640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcdbc0678c0 0x7fcdbc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.498+0000 7fcdd656d640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcdbc0678c0 0x7fcdbc069d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00ae000 tx=0x7fcdc0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:30.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.502+0000 7fcdceffd640 1 -- 172.21.15.52:0/1728788536 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcdc8022070 con 0x7fcdd015c700 2024-05-04T12:18:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.654+0000 7fcdd87d1640 1 -- 172.21.15.52:0/1728788536 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcdd0075320 con 0x7fcdd015c700 2024-05-04T12:18:30.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.658+0000 7fcdceffd640 1 -- 172.21.15.52:0/1728788536 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fcdc8010040 con 0x7fcdd015c700 2024-05-04T12:18:30.659 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:30.660 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.658+0000 7fcdd87d1640 1 -- 172.21.15.52:0/1728788536 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcdbc0678c0 msgr2=0x7fcdbc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.658+0000 7fcdd87d1640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcdbc0678c0 0x7fcdbc069d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00ae000 tx=0x7fcdc0008040 comp rx=0 tx=0).stop 2024-05-04T12:18:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.658+0000 7fcdd87d1640 1 -- 172.21.15.52:0/1728788536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd015c700 msgr2=0x7fcdd01390f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.658+0000 7fcdd87d1640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd015c700 0x7fcdd01390f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009d30 tx=0x7fcdc800c490 comp rx=0 tx=0).stop 2024-05-04T12:18:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.658+0000 7fcdd87d1640 1 -- 172.21.15.52:0/1728788536 shutdown_connections 2024-05-04T12:18:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.658+0000 7fcdd87d1640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcdbc0678c0 0x7fcdbc069d80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.658+0000 7fcdd87d1640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd015c700 0x7fcdd01390f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.658+0000 7fcdd87d1640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcdd01417d0 0x7fcdd015eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.658+0000 7fcdd87d1640 1 --2- 172.21.15.52:0/1728788536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdd00cd5c0 0x7fcdd013b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.658+0000 7fcdd87d1640 1 -- 172.21.15.52:0/1728788536 >> 172.21.15.52:0/1728788536 conn(0x7fcdd00b2380 msgr2=0x7fcdd006e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.658+0000 7fcdd87d1640 1 -- 172.21.15.52:0/1728788536 shutdown_connections 2024-05-04T12:18:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.658+0000 7fcdd87d1640 1 -- 172.21.15.52:0/1728788536 wait complete. 2024-05-04T12:18:30.677 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:18:30.678 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:18:30.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc605d29640 1 Processor -- start 2024-05-04T12:18:30.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc605d29640 1 -- start start 2024-05-04T12:18:30.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc605d29640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc60015e180 0x7fc6000ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc605d29640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6000ce6a0 con 0x7fc600162660 2024-05-04T12:18:30.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc605d29640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6000ce980 con 0x7fc600070de0 2024-05-04T12:18:30.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc605d29640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc6000cf220 con 0x7fc60015e180 2024-05-04T12:18:30.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5feffd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc600162660 0x7fc6000706d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51608/0 (socket says 172.21.15.52:51608) 2024-05-04T12:18:30.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5feffd640 1 -- 172.21.15.52:0/841389218 learned_addr learned my addr 172.21.15.52:0/841389218 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:30.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5ff7fe640 1 --2- 172.21.15.52:0/841389218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc60015e180 0x7fc6000ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 -- 172.21.15.52:0/841389218 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2223413010 0 0) 0x7fc6000ce6a0 con 0x7fc600162660 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 -- 172.21.15.52:0/841389218 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5e8003530 con 0x7fc600162660 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 -- 172.21.15.52:0/841389218 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3809927659 0 0) 0x7fc5e8003530 con 0x7fc600162660 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 -- 172.21.15.52:0/841389218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc60015e180 msgr2=0x7fc6000ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 --2- 172.21.15.52:0/841389218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc60015e180 0x7fc6000ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 -- 172.21.15.52:0/841389218 >> v1:172.21.15.111:6789/0 conn(0x7fc600070de0 legacy=0x7fc6000cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 -- 172.21.15.52:0/841389218 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6000cec60 con 0x7fc600162660 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 -- 172.21.15.52:0/841389218 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7fc5f4001680 con 0x7fc600162660 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 -- 172.21.15.52:0/841389218 >> v1:172.21.15.52:6789/0 conn(0x7fc600162660 legacy=0x7fc6000706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 --2- 172.21.15.52:0/841389218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e80039f0 0x7fc5e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 --2- 172.21.15.52:0/841389218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc60015e180 0x7fc5e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 --2- 172.21.15.52:0/841389218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e8004d50 0x7fc5e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 -- 172.21.15.52:0/841389218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e800ba80 con 0x7fc5e80039f0 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 -- 172.21.15.52:0/841389218 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e800bc00 con 0x7fc5e8004d50 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fe7fc640 1 -- 172.21.15.52:0/841389218 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e800bdb0 con 0x7fc60015e180 2024-05-04T12:18:30.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5ff7fe640 1 --2- 172.21.15.52:0/841389218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e80039f0 0x7fc5e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5feffd640 1 --2- 172.21.15.52:0/841389218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc60015e180 0x7fc5e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fffff640 1 --2- 172.21.15.52:0/841389218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e8004d50 0x7fc5e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5ff7fe640 1 -- 172.21.15.52:0/841389218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc60015e180 msgr2=0x7fc5e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:30.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5ff7fe640 1 --2- 172.21.15.52:0/841389218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc60015e180 0x7fc5e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5ff7fe640 1 -- 172.21.15.52:0/841389218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e8004d50 msgr2=0x7fc5e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:30.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5ff7fe640 1 --2- 172.21.15.52:0/841389218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e8004d50 0x7fc5e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5ff7fe640 1 -- 172.21.15.52:0/841389218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e8003530 con 0x7fc5e80039f0 2024-05-04T12:18:30.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.802+0000 7fc5fffff640 1 --2- 172.21.15.52:0/841389218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e8004d50 0x7fc5e80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:30.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc5ff7fe640 1 --2- 172.21.15.52:0/841389218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e80039f0 0x7fc5e8003dd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009ae0 tx=0x7fc5f001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c011005eb81dcc51 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:30.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc5fe7fc640 1 -- 172.21.15.52:0/841389218 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f00b2070 con 0x7fc5e80039f0 2024-05-04T12:18:30.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc5fe7fc640 1 -- 172.21.15.52:0/841389218 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f00a5660 con 0x7fc5e80039f0 2024-05-04T12:18:30.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc5fe7fc640 1 -- 172.21.15.52:0/841389218 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f00096b0 con 0x7fc5e80039f0 2024-05-04T12:18:30.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc605d29640 1 -- 172.21.15.52:0/841389218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e80039f0 msgr2=0x7fc5e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc605d29640 1 --2- 172.21.15.52:0/841389218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e80039f0 0x7fc5e8003dd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009ae0 tx=0x7fc5f001ca70 comp rx=0 tx=0).stop 2024-05-04T12:18:30.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc5fffff640 1 -- 172.21.15.52:0/841389218 reap_dead start 2024-05-04T12:18:30.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc605d29640 1 -- 172.21.15.52:0/841389218 shutdown_connections 2024-05-04T12:18:30.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc605d29640 1 -- 172.21.15.52:0/841389218 >> 172.21.15.52:0/841389218 conn(0x7fc6000d12c0 msgr2=0x7fc6000d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:30.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc605d29640 1 -- 172.21.15.52:0/841389218 shutdown_connections 2024-05-04T12:18:30.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc605d29640 1 -- 172.21.15.52:0/841389218 wait complete. 2024-05-04T12:18:30.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc605d29640 1 Processor -- start 2024-05-04T12:18:30.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc605d29640 1 -- start start 2024-05-04T12:18:30.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc605d29640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e80039f0 0x7fc600147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc605d29640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6001604c0 0x7fc600159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc605d29640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc600160940 0x7fc60015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc605d29640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc6000ce0c0 con 0x7fc5e80039f0 2024-05-04T12:18:30.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc605d29640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc600162ac0 con 0x7fc600160940 2024-05-04T12:18:30.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc605d29640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc6000ce240 con 0x7fc6001604c0 2024-05-04T12:18:30.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc5ff7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e80039f0 0x7fc600147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc5ff7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e80039f0 0x7fc600147d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53692/0 (socket says 172.21.15.52:53692) 2024-05-04T12:18:30.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.806+0000 7fc5ff7fe640 1 -- 172.21.15.52:0/2916001810 learned_addr learned my addr 172.21.15.52:0/2916001810 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:30.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5fffff640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc600160940 0x7fc60015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5feffd640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6001604c0 0x7fc600159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5ff7fe640 1 -- 172.21.15.52:0/2916001810 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6001604c0 msgr2=0x7fc600159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5ff7fe640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6001604c0 0x7fc600159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5ff7fe640 1 -- 172.21.15.52:0/2916001810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc600160940 msgr2=0x7fc60015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5ff7fe640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc600160940 0x7fc60015a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5ff7fe640 1 -- 172.21.15.52:0/2916001810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc600148260 con 0x7fc5e80039f0 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5feffd640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6001604c0 0x7fc600159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5ff7fe640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e80039f0 0x7fc600147d20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc5f001c8a0 tx=0x7fc5e8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5e7fff640 1 -- 172.21.15.52:0/2916001810 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f00b2040 con 0x7fc5e80039f0 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5e7fff640 1 -- 172.21.15.52:0/2916001810 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f000f000 con 0x7fc5e80039f0 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc605d29640 1 -- 172.21.15.52:0/2916001810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60015f3b0 con 0x7fc5e80039f0 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5e7fff640 1 -- 172.21.15.52:0/2916001810 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0004220 con 0x7fc5e80039f0 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc605d29640 1 -- 172.21.15.52:0/2916001810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60015f9e0 con 0x7fc5e80039f0 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc605d29640 1 -- 172.21.15.52:0/2916001810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc600062e50 con 0x7fc5e80039f0 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5e7fff640 1 -- 172.21.15.52:0/2916001810 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc5f00be030 con 0x7fc5e80039f0 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5e7fff640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5ec0678c0 0x7fc5ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:30.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5e7fff640 1 -- 172.21.15.52:0/2916001810 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc5f01201b0 con 0x7fc5e80039f0 2024-05-04T12:18:30.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5feffd640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5ec0678c0 0x7fc5ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:30.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.810+0000 7fc5feffd640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5ec0678c0 0x7fc5ec069d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4001c20 tx=0x7fc5f4000f60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:30.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.814+0000 7fc5e7fff640 1 -- 172.21.15.52:0/2916001810 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc5f00c5020 con 0x7fc5e80039f0 2024-05-04T12:18:30.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.978+0000 7fc605d29640 1 -- 172.21.15.52:0/2916001810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc600075320 con 0x7fc5e80039f0 2024-05-04T12:18:30.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.978+0000 7fc5e7fff640 1 -- 172.21.15.52:0/2916001810 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fc5f0016780 con 0x7fc5e80039f0 2024-05-04T12:18:30.981 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:18:30.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.982+0000 7fc605d29640 1 -- 172.21.15.52:0/2916001810 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5ec0678c0 msgr2=0x7fc5ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.982+0000 7fc605d29640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5ec0678c0 0x7fc5ec069d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4001c20 tx=0x7fc5f4000f60 comp rx=0 tx=0).stop 2024-05-04T12:18:30.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.982+0000 7fc605d29640 1 -- 172.21.15.52:0/2916001810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e80039f0 msgr2=0x7fc600147d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:30.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.982+0000 7fc605d29640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e80039f0 0x7fc600147d20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc5f001c8a0 tx=0x7fc5e8005840 comp rx=0 tx=0).stop 2024-05-04T12:18:30.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.982+0000 7fc605d29640 1 -- 172.21.15.52:0/2916001810 shutdown_connections 2024-05-04T12:18:30.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.982+0000 7fc605d29640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5ec0678c0 0x7fc5ec069d80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.982+0000 7fc605d29640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc600160940 0x7fc60015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.982+0000 7fc605d29640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6001604c0 0x7fc600159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.982+0000 7fc605d29640 1 --2- 172.21.15.52:0/2916001810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e80039f0 0x7fc600147d20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:30.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.982+0000 7fc605d29640 1 -- 172.21.15.52:0/2916001810 >> 172.21.15.52:0/2916001810 conn(0x7fc6000d12c0 msgr2=0x7fc60006e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:30.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.982+0000 7fc605d29640 1 -- 172.21.15.52:0/2916001810 shutdown_connections 2024-05-04T12:18:30.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:30.982+0000 7fc605d29640 1 -- 172.21.15.52:0/2916001810 wait complete. 2024-05-04T12:18:31.000 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:18:31.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67df2c4640 1 Processor -- start 2024-05-04T12:18:31.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67df2c4640 1 -- start start 2024-05-04T12:18:31.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67df2c4640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815bd90 0x7f67d80cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67df2c4640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67d80ccb80 con 0x7f67d806fa80 2024-05-04T12:18:31.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67df2c4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67d80cce60 con 0x7f67d815c170 2024-05-04T12:18:31.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67df2c4640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f67d80cc930 con 0x7f67d815bd90 2024-05-04T12:18:31.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67dd861640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f67d806fa80 0x7f67d806cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51618/0 (socket says 172.21.15.52:51618) 2024-05-04T12:18:31.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67dd861640 1 -- 172.21.15.52:0/2070088972 learned_addr learned my addr 172.21.15.52:0/2070088972 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:31.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67dd060640 1 --2- 172.21.15.52:0/2070088972 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815bd90 0x7f67d80cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 -- 172.21.15.52:0/2070088972 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2979294509 0 0) 0x7f67d80ccb80 con 0x7f67d806fa80 2024-05-04T12:18:31.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 -- 172.21.15.52:0/2070088972 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67c4005740 con 0x7f67d806fa80 2024-05-04T12:18:31.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 -- 172.21.15.52:0/2070088972 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2074313474 0 0) 0x7f67c4005740 con 0x7f67d806fa80 2024-05-04T12:18:31.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 -- 172.21.15.52:0/2070088972 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815bd90 msgr2=0x7f67d80cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:31.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 --2- 172.21.15.52:0/2070088972 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815bd90 0x7f67d80cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 -- 172.21.15.52:0/2070088972 >> v1:172.21.15.111:6789/0 conn(0x7f67d815c170 legacy=0x7f67d806f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 -- 172.21.15.52:0/2070088972 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d80cd140 con 0x7f67d806fa80 2024-05-04T12:18:31.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 -- 172.21.15.52:0/2070088972 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f67d0001680 con 0x7f67d806fa80 2024-05-04T12:18:31.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67dd060640 1 --2- 172.21.15.52:0/2070088972 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815bd90 0x7f67d80cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:18:31.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 -- 172.21.15.52:0/2070088972 >> v1:172.21.15.52:6789/0 conn(0x7f67d806fa80 legacy=0x7f67d806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 --2- 172.21.15.52:0/2070088972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67c4005c00 0x7f67c4006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 --2- 172.21.15.52:0/2070088972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67c4006a20 0x7f67c4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 --2- 172.21.15.52:0/2070088972 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815bd90 0x7f67c4007450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 -- 172.21.15.52:0/2070088972 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f67d80cc930 con 0x7f67c4006a20 2024-05-04T12:18:31.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 -- 172.21.15.52:0/2070088972 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f67c400dd70 con 0x7f67c4005c00 2024-05-04T12:18:31.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67cffff640 1 -- 172.21.15.52:0/2070088972 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f67c400df40 con 0x7f67d815bd90 2024-05-04T12:18:31.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67dd060640 1 --2- 172.21.15.52:0/2070088972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67c4005c00 0x7f67c4006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67dd861640 1 --2- 172.21.15.52:0/2070088972 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815bd90 0x7f67c4007450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67dc85f640 1 --2- 172.21.15.52:0/2070088972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67c4006a20 0x7f67c4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67dd060640 1 -- 172.21.15.52:0/2070088972 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815bd90 msgr2=0x7f67c4007450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:31.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67dd060640 1 --2- 172.21.15.52:0/2070088972 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815bd90 0x7f67c4007450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67dd060640 1 -- 172.21.15.52:0/2070088972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67c4006a20 msgr2=0x7f67c4006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:31.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67dd060640 1 --2- 172.21.15.52:0/2070088972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67c4006a20 0x7f67c4006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.126+0000 7f67dd060640 1 -- 172.21.15.52:0/2070088972 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c4005740 con 0x7f67c4005c00 2024-05-04T12:18:31.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.130+0000 7f67dd060640 1 --2- 172.21.15.52:0/2070088972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67c4005c00 0x7f67c4006040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f67d400e320 tx=0x7f67d401cdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bd00e2d2165d03f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:31.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.130+0000 7f67cffff640 1 -- 172.21.15.52:0/2070088972 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40b2070 con 0x7f67c4005c00 2024-05-04T12:18:31.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.130+0000 7f67cffff640 1 -- 172.21.15.52:0/2070088972 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67d40a5cc0 con 0x7f67c4005c00 2024-05-04T12:18:31.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.130+0000 7f67cffff640 1 -- 172.21.15.52:0/2070088972 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4009de0 con 0x7f67c4005c00 2024-05-04T12:18:31.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.130+0000 7f67df2c4640 1 -- 172.21.15.52:0/2070088972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67c4005c00 msgr2=0x7f67c4006040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.130+0000 7f67df2c4640 1 --2- 172.21.15.52:0/2070088972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67c4005c00 0x7f67c4006040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f67d400e320 tx=0x7f67d401cdc0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.130+0000 7f67dd861640 1 -- 172.21.15.52:0/2070088972 reap_dead start 2024-05-04T12:18:31.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.130+0000 7f67df2c4640 1 -- 172.21.15.52:0/2070088972 shutdown_connections 2024-05-04T12:18:31.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.130+0000 7f67df2c4640 1 -- 172.21.15.52:0/2070088972 >> 172.21.15.52:0/2070088972 conn(0x7f67d8067730 msgr2=0x7f67d80b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:31.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.130+0000 7f67df2c4640 1 -- 172.21.15.52:0/2070088972 shutdown_connections 2024-05-04T12:18:31.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.130+0000 7f67df2c4640 1 -- 172.21.15.52:0/2070088972 wait complete. 2024-05-04T12:18:31.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67df2c4640 1 Processor -- start 2024-05-04T12:18:31.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67df2c4640 1 -- start start 2024-05-04T12:18:31.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67df2c4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67c4005c00 0x7f67d8154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67df2c4640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815d4f0 0x7f67d813df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67df2c4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67d815d970 0x7f67d813e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67df2c4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f67d80ceb30 con 0x7f67d815d970 2024-05-04T12:18:31.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67df2c4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f67d80ce9b0 con 0x7f67c4005c00 2024-05-04T12:18:31.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67df2c4640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f67d80cecb0 con 0x7f67d815d4f0 2024-05-04T12:18:31.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67dd861640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67d815d970 0x7f67d813e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67dd861640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67d815d970 0x7f67d813e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53714/0 (socket says 172.21.15.52:53714) 2024-05-04T12:18:31.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67dd861640 1 -- 172.21.15.52:0/320478595 learned_addr learned my addr 172.21.15.52:0/320478595 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:31.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67dd060640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67c4005c00 0x7f67d8154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67dc85f640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815d4f0 0x7f67d813df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67dd861640 1 -- 172.21.15.52:0/320478595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815d4f0 msgr2=0x7f67d813df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67dd861640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815d4f0 0x7f67d813df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67dd861640 1 -- 172.21.15.52:0/320478595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67c4005c00 msgr2=0x7f67d8154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67dd861640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67c4005c00 0x7f67d8154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67dd861640 1 -- 172.21.15.52:0/320478595 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d81fc400 con 0x7f67d815d970 2024-05-04T12:18:31.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67dc85f640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815d4f0 0x7f67d813df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:31.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67dd861640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67d815d970 0x7f67d813e4c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f67d8069ab0 tx=0x7f67d0007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:31.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67cdffb640 1 -- 172.21.15.52:0/320478595 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d000f700 con 0x7f67d815d970 2024-05-04T12:18:31.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67cdffb640 1 -- 172.21.15.52:0/320478595 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67d0018070 con 0x7f67d815d970 2024-05-04T12:18:31.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67df2c4640 1 -- 172.21.15.52:0/320478595 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d8141c60 con 0x7f67d815d970 2024-05-04T12:18:31.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67cdffb640 1 -- 172.21.15.52:0/320478595 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d000fae0 con 0x7f67d815d970 2024-05-04T12:18:31.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.134+0000 7f67df2c4640 1 -- 172.21.15.52:0/320478595 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d81545b0 con 0x7f67d815d970 2024-05-04T12:18:31.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.138+0000 7f67df2c4640 1 -- 172.21.15.52:0/320478595 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d8062e50 con 0x7f67d815d970 2024-05-04T12:18:31.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.138+0000 7f67cdffb640 1 -- 172.21.15.52:0/320478595 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f67d000fc80 con 0x7f67d815d970 2024-05-04T12:18:31.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.138+0000 7f67cdffb640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67bc0678f0 0x7f67bc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.138+0000 7f67cdffb640 1 -- 172.21.15.52:0/320478595 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f67d0085940 con 0x7f67d815d970 2024-05-04T12:18:31.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.138+0000 7f67dd060640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67bc0678f0 0x7f67bc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.138+0000 7f67dd060640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67bc0678f0 0x7f67bc069db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f67d40af000 tx=0x7f67d4004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:31.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.142+0000 7f67cdffb640 1 -- 172.21.15.52:0/320478595 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f67d0056480 con 0x7f67d815d970 2024-05-04T12:18:31.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.302+0000 7f67df2c4640 1 -- 172.21.15.52:0/320478595 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f67d80b1860 con 0x7f67d815d970 2024-05-04T12:18:31.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.306+0000 7f67cdffb640 1 -- 172.21.15.52:0/320478595 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f67d0055d80 con 0x7f67d815d970 2024-05-04T12:18:31.307 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:18:31.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.306+0000 7f67df2c4640 1 -- 172.21.15.52:0/320478595 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67bc0678f0 msgr2=0x7f67bc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.306+0000 7f67df2c4640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67bc0678f0 0x7f67bc069db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f67d40af000 tx=0x7f67d4004ac0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.306+0000 7f67df2c4640 1 -- 172.21.15.52:0/320478595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67d815d970 msgr2=0x7f67d813e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.306+0000 7f67df2c4640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67d815d970 0x7f67d813e4c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f67d8069ab0 tx=0x7f67d0007430 comp rx=0 tx=0).stop 2024-05-04T12:18:31.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.306+0000 7f67df2c4640 1 -- 172.21.15.52:0/320478595 shutdown_connections 2024-05-04T12:18:31.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.306+0000 7f67df2c4640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67bc0678f0 0x7f67bc069db0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.306+0000 7f67df2c4640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67d815d970 0x7f67d813e4c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.306+0000 7f67df2c4640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67d815d4f0 0x7f67d813df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.306+0000 7f67df2c4640 1 --2- 172.21.15.52:0/320478595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67c4005c00 0x7f67d8154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.306+0000 7f67df2c4640 1 -- 172.21.15.52:0/320478595 >> 172.21.15.52:0/320478595 conn(0x7f67d8067730 msgr2=0x7f67d80ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:31.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.306+0000 7f67df2c4640 1 -- 172.21.15.52:0/320478595 shutdown_connections 2024-05-04T12:18:31.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.306+0000 7f67df2c4640 1 -- 172.21.15.52:0/320478595 wait complete. 2024-05-04T12:18:31.326 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:18:31.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e2ce27640 1 Processor -- start 2024-05-04T12:18:31.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e2ce27640 1 -- start start 2024-05-04T12:18:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e2ce27640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7e2815c590 0x7f7e280cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e2ce27640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7e280cccd0 0x7f7e2806e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e2ce27640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e2806dbf0 con 0x7f7e28145960 2024-05-04T12:18:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e2ce27640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f7e280cd140 con 0x7f7e280cccd0 2024-05-04T12:18:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e2ce27640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f7e2806cfe0 con 0x7f7e2815c590 2024-05-04T12:18:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e26d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7e280cccd0 0x7f7e2806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e2659c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f7e28145960 0x7f7e28160590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51632/0 (socket says 172.21.15.52:51632) 2024-05-04T12:18:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e2659c640 1 -- 172.21.15.52:0/1318153577 learned_addr learned my addr 172.21.15.52:0/1318153577 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e25d9b640 1 --2- 172.21.15.52:0/1318153577 >> v2:172.21.15.111:3300/0 conn(0x7f7e2815c590 0x7f7e280cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e26d9d640 1 -- 172.21.15.52:0/1318153577 >> v2:172.21.15.111:3300/0 conn(0x7f7e2815c590 msgr2=0x7f7e280cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e26d9d640 1 --2- 172.21.15.52:0/1318153577 >> v2:172.21.15.111:3300/0 conn(0x7f7e2815c590 0x7f7e280cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e26d9d640 1 -- 172.21.15.52:0/1318153577 >> v1:172.21.15.52:6789/0 conn(0x7f7e28145960 legacy=0x7f7e28160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e2559a640 1 -- 172.21.15.52:0/1318153577 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1111181050 0 0) 0x7f7e2806dbf0 con 0x7f7e28145960 2024-05-04T12:18:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.454+0000 7f7e26d9d640 1 -- 172.21.15.52:0/1318153577 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2806d2c0 con 0x7f7e280cccd0 2024-05-04T12:18:31.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e26d9d640 1 --2- 172.21.15.52:0/1318153577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e280cccd0 0x7f7e2806e080 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7e280b21a0 tx=0x7f7e2001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d482271c08edc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:31.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2559a640 1 -- 172.21.15.52:0/1318153577 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e200b2070 con 0x7f7e280cccd0 2024-05-04T12:18:31.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2559a640 1 -- 172.21.15.52:0/1318153577 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e200a5930 con 0x7f7e280cccd0 2024-05-04T12:18:31.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2559a640 1 -- 172.21.15.52:0/1318153577 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e200a5d00 con 0x7f7e280cccd0 2024-05-04T12:18:31.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 -- 172.21.15.52:0/1318153577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e280cccd0 msgr2=0x7f7e2806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 --2- 172.21.15.52:0/1318153577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e280cccd0 0x7f7e2806e080 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7e280b21a0 tx=0x7f7e2001c870 comp rx=0 tx=0).stop 2024-05-04T12:18:31.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 -- 172.21.15.52:0/1318153577 shutdown_connections 2024-05-04T12:18:31.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 --2- 172.21.15.52:0/1318153577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e280cccd0 0x7f7e2806e080 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 --2- 172.21.15.52:0/1318153577 >> v2:172.21.15.111:3300/0 conn(0x7f7e2815c590 0x7f7e280cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 -- 172.21.15.52:0/1318153577 >> 172.21.15.52:0/1318153577 conn(0x7f7e28067730 msgr2=0x7f7e280b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:31.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 -- 172.21.15.52:0/1318153577 shutdown_connections 2024-05-04T12:18:31.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 -- 172.21.15.52:0/1318153577 wait complete. 2024-05-04T12:18:31.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 Processor -- start 2024-05-04T12:18:31.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 -- start start 2024-05-04T12:18:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e280cccd0 0x7f7e28132b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e28145960 0x7f7e2815bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e2815c590 0x7f7e28165390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2806cdc0 con 0x7f7e28145960 2024-05-04T12:18:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2806cc40 con 0x7f7e280cccd0 2024-05-04T12:18:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2ce27640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f7e28145d40 con 0x7f7e2815c590 2024-05-04T12:18:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e25d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e28145960 0x7f7e2815bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e25d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e28145960 0x7f7e2815bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53728/0 (socket says 172.21.15.52:53728) 2024-05-04T12:18:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e25d9b640 1 -- 172.21.15.52:0/2114851723 learned_addr learned my addr 172.21.15.52:0/2114851723 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e2659c640 1 --2- 172.21.15.52:0/2114851723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e280cccd0 0x7f7e28132b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e26d9d640 1 --2- 172.21.15.52:0/2114851723 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e2815c590 0x7f7e28165390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e25d9b640 1 -- 172.21.15.52:0/2114851723 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e2815c590 msgr2=0x7f7e28165390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e25d9b640 1 --2- 172.21.15.52:0/2114851723 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e2815c590 0x7f7e28165390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e25d9b640 1 -- 172.21.15.52:0/2114851723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e280cccd0 msgr2=0x7f7e28132b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e25d9b640 1 --2- 172.21.15.52:0/2114851723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e280cccd0 0x7f7e28132b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.458+0000 7f7e25d9b640 1 -- 172.21.15.52:0/2114851723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e281333c0 con 0x7f7e28145960 2024-05-04T12:18:31.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.462+0000 7f7e25d9b640 1 --2- 172.21.15.52:0/2114851723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e28145960 0x7f7e2815bbc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00d5f0 tx=0x7f7e1c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:31.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.462+0000 7f7e16ffd640 1 -- 172.21.15.52:0/2114851723 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c0125f0 con 0x7f7e28145960 2024-05-04T12:18:31.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.462+0000 7f7e16ffd640 1 -- 172.21.15.52:0/2114851723 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e1c019070 con 0x7f7e28145960 2024-05-04T12:18:31.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.462+0000 7f7e2ce27640 1 -- 172.21.15.52:0/2114851723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e281336a0 con 0x7f7e28145960 2024-05-04T12:18:31.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.462+0000 7f7e16ffd640 1 -- 172.21.15.52:0/2114851723 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c0129b0 con 0x7f7e28145960 2024-05-04T12:18:31.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.462+0000 7f7e2ce27640 1 -- 172.21.15.52:0/2114851723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e28133c60 con 0x7f7e28145960 2024-05-04T12:18:31.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.462+0000 7f7e2ce27640 1 -- 172.21.15.52:0/2114851723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e28062e50 con 0x7f7e28145960 2024-05-04T12:18:31.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.462+0000 7f7e16ffd640 1 -- 172.21.15.52:0/2114851723 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7e1c012b50 con 0x7f7e28145960 2024-05-04T12:18:31.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.462+0000 7f7e16ffd640 1 --2- 172.21.15.52:0/2114851723 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7e080678f0 0x7f7e08069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.462+0000 7f7e16ffd640 1 -- 172.21.15.52:0/2114851723 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7e1c086080 con 0x7f7e28145960 2024-05-04T12:18:31.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.462+0000 7f7e2659c640 1 --2- 172.21.15.52:0/2114851723 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7e080678f0 0x7f7e08069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.462+0000 7f7e2659c640 1 --2- 172.21.15.52:0/2114851723 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7e080678f0 0x7f7e08069db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7e18000f70 tx=0x7f7e18007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:31.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.466+0000 7f7e16ffd640 1 -- 172.21.15.52:0/2114851723 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7e1c056bc0 con 0x7f7e28145960 2024-05-04T12:18:31.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.626+0000 7f7e2ce27640 1 -- 172.21.15.52:0/2114851723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7e28075320 con 0x7f7e28145960 2024-05-04T12:18:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.626+0000 7f7e16ffd640 1 -- 172.21.15.52:0/2114851723 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f7e1c0564c0 con 0x7f7e28145960 2024-05-04T12:18:31.629 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:18:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.630+0000 7f7e2ce27640 1 -- 172.21.15.52:0/2114851723 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7e080678f0 msgr2=0x7f7e08069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.630+0000 7f7e2ce27640 1 --2- 172.21.15.52:0/2114851723 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7e080678f0 0x7f7e08069db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7e18000f70 tx=0x7f7e18007c10 comp rx=0 tx=0).stop 2024-05-04T12:18:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.630+0000 7f7e2ce27640 1 -- 172.21.15.52:0/2114851723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e28145960 msgr2=0x7f7e2815bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.630+0000 7f7e2ce27640 1 --2- 172.21.15.52:0/2114851723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e28145960 0x7f7e2815bbc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00d5f0 tx=0x7f7e1c00dab0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.630+0000 7f7e2ce27640 1 -- 172.21.15.52:0/2114851723 shutdown_connections 2024-05-04T12:18:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.630+0000 7f7e2ce27640 1 --2- 172.21.15.52:0/2114851723 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7e080678f0 0x7f7e08069db0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.630+0000 7f7e2ce27640 1 --2- 172.21.15.52:0/2114851723 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e2815c590 0x7f7e28165390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.630+0000 7f7e2ce27640 1 --2- 172.21.15.52:0/2114851723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e28145960 0x7f7e2815bbc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.630+0000 7f7e2ce27640 1 --2- 172.21.15.52:0/2114851723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e280cccd0 0x7f7e28132b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.630+0000 7f7e2ce27640 1 -- 172.21.15.52:0/2114851723 >> 172.21.15.52:0/2114851723 conn(0x7f7e28067730 msgr2=0x7f7e2813b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:31.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.630+0000 7f7e2ce27640 1 -- 172.21.15.52:0/2114851723 shutdown_connections 2024-05-04T12:18:31.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.630+0000 7f7e2ce27640 1 -- 172.21.15.52:0/2114851723 wait complete. 2024-05-04T12:18:31.649 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T12:18:31.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f369421b640 1 Processor -- start 2024-05-04T12:18:31.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f369421b640 1 -- start start 2024-05-04T12:18:31.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f369421b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f368c06db40 0x7f368c067070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f369421b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f368c06f290 0x7f368c071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f369421b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f368c0cea70 con 0x7f368c15ae50 2024-05-04T12:18:31.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f369421b640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f368c06f700 con 0x7f368c06db40 2024-05-04T12:18:31.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f369421b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f368c067770 con 0x7f368c06f290 2024-05-04T12:18:31.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f36917b6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f368c06db40 0x7f368c067070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f36917b6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f368c06db40 0x7f368c067070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53738/0 (socket says 172.21.15.52:53738) 2024-05-04T12:18:31.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f36917b6640 1 -- 172.21.15.52:0/3487034864 learned_addr learned my addr 172.21.15.52:0/3487034864 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:31.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f36927b8640 1 --2- 172.21.15.52:0/3487034864 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f368c06f290 0x7f368c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f36917b6640 1 -- 172.21.15.52:0/3487034864 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f368c06f290 msgr2=0x7f368c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:31.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f36917b6640 1 --2- 172.21.15.52:0/3487034864 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f368c06f290 0x7f368c071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f36917b6640 1 -- 172.21.15.52:0/3487034864 >> v1:172.21.15.111:6789/0 conn(0x7f368c15ae50 legacy=0x7f368c06d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:31.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f36917b6640 1 -- 172.21.15.52:0/3487034864 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c0ced10 con 0x7f368c06db40 2024-05-04T12:18:31.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f36927b8640 1 --2- 172.21.15.52:0/3487034864 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f368c06f290 0x7f368c071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:31.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f36917b6640 1 --2- 172.21.15.52:0/3487034864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f368c06db40 0x7f368c067070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f367c009870 tx=0x7f367c01e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73a24d957ad0e92 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:31.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.782+0000 7f3690fb5640 1 -- 172.21.15.52:0/3487034864 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c0aa630 con 0x7f368c06db40 2024-05-04T12:18:31.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f3690fb5640 1 -- 172.21.15.52:0/3487034864 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f367c0aa7d0 con 0x7f368c06db40 2024-05-04T12:18:31.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f3690fb5640 1 -- 172.21.15.52:0/3487034864 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c0aaba0 con 0x7f368c06db40 2024-05-04T12:18:31.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 -- 172.21.15.52:0/3487034864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f368c06db40 msgr2=0x7f368c067070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 --2- 172.21.15.52:0/3487034864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f368c06db40 0x7f368c067070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f367c009870 tx=0x7f367c01e730 comp rx=0 tx=0).stop 2024-05-04T12:18:31.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 -- 172.21.15.52:0/3487034864 shutdown_connections 2024-05-04T12:18:31.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 --2- 172.21.15.52:0/3487034864 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f368c06f290 0x7f368c071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 --2- 172.21.15.52:0/3487034864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f368c06db40 0x7f368c067070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 -- 172.21.15.52:0/3487034864 >> 172.21.15.52:0/3487034864 conn(0x7f368c0b3190 msgr2=0x7f368c0b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:31.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 -- 172.21.15.52:0/3487034864 shutdown_connections 2024-05-04T12:18:31.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 -- 172.21.15.52:0/3487034864 wait complete. 2024-05-04T12:18:31.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 Processor -- start 2024-05-04T12:18:31.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 -- start start 2024-05-04T12:18:31.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f368c06db40 0x7f368c14b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f368c06f290 0x7f368c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f368c15ae50 0x7f368c139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f368c0d0180 con 0x7f368c06db40 2024-05-04T12:18:31.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f368c0d0000 con 0x7f368c15ae50 2024-05-04T12:18:31.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f369421b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f368c0d0300 con 0x7f368c06f290 2024-05-04T12:18:31.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f3691fb7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f368c06db40 0x7f368c14b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f3691fb7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f368c06db40 0x7f368c14b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53754/0 (socket says 172.21.15.52:53754) 2024-05-04T12:18:31.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f3691fb7640 1 -- 172.21.15.52:0/873646240 learned_addr learned my addr 172.21.15.52:0/873646240 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:31.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f36917b6640 1 --2- 172.21.15.52:0/873646240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f368c06f290 0x7f368c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f36927b8640 1 --2- 172.21.15.52:0/873646240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f368c15ae50 0x7f368c139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f3691fb7640 1 -- 172.21.15.52:0/873646240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f368c06f290 msgr2=0x7f368c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f3691fb7640 1 --2- 172.21.15.52:0/873646240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f368c06f290 0x7f368c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f3691fb7640 1 -- 172.21.15.52:0/873646240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f368c15ae50 msgr2=0x7f368c139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f3691fb7640 1 --2- 172.21.15.52:0/873646240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f368c15ae50 0x7f368c139e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:31.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.786+0000 7f3691fb7640 1 -- 172.21.15.52:0/873646240 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c14be90 con 0x7f368c06db40 2024-05-04T12:18:31.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.790+0000 7f3691fb7640 1 --2- 172.21.15.52:0/873646240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f368c06db40 0x7f368c14b740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f368800b440 tx=0x7f368800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:31.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.790+0000 7f36827fc640 1 -- 172.21.15.52:0/873646240 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688015020 con 0x7f368c06db40 2024-05-04T12:18:31.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.790+0000 7f36827fc640 1 -- 172.21.15.52:0/873646240 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3688011890 con 0x7f368c06db40 2024-05-04T12:18:31.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.790+0000 7f369421b640 1 -- 172.21.15.52:0/873646240 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368c14c170 con 0x7f368c06db40 2024-05-04T12:18:31.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.790+0000 7f36827fc640 1 -- 172.21.15.52:0/873646240 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688011c80 con 0x7f368c06db40 2024-05-04T12:18:31.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.790+0000 7f369421b640 1 -- 172.21.15.52:0/873646240 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368c14c730 con 0x7f368c06db40 2024-05-04T12:18:31.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.790+0000 7f369421b640 1 -- 172.21.15.52:0/873646240 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f368c062e50 con 0x7f368c06db40 2024-05-04T12:18:31.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.790+0000 7f36827fc640 1 -- 172.21.15.52:0/873646240 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3688018020 con 0x7f368c06db40 2024-05-04T12:18:31.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.790+0000 7f36827fc640 1 --2- 172.21.15.52:0/873646240 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f36700678c0 0x7f3670069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:31.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.790+0000 7f36917b6640 1 --2- 172.21.15.52:0/873646240 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f36700678c0 0x7f3670069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:31.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.790+0000 7f36827fc640 1 -- 172.21.15.52:0/873646240 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3688085b60 con 0x7f368c06db40 2024-05-04T12:18:31.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.790+0000 7f36917b6640 1 --2- 172.21.15.52:0/873646240 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f36700678c0 0x7f3670069d80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f367c0b1000 tx=0x7f367c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:31.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.794+0000 7f36827fc640 1 -- 172.21.15.52:0/873646240 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f368801f030 con 0x7f368c06db40 2024-05-04T12:18:31.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.994+0000 7f369421b640 1 -- 172.21.15.52:0/873646240 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f368c075320 con 0x7f368c06db40 2024-05-04T12:18:31.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.994+0000 7f36827fc640 1 -- 172.21.15.52:0/873646240 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7f368800d040 con 0x7f368c06db40 2024-05-04T12:18:31.997 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:31.997 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6836/482177111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":482177111},{"type":"v1","addr":"172.21.15.52:6836","nonce":482177111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/163453709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6835","nonce":163453709},{"type":"v1","addr":"172.21.15.52:6837","nonce":163453709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3596668218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3596668218},{"type":"v1","addr":"172.21.15.111:6833","nonce":3596668218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-04T12:18:22.189525+0000","modified":"2024-05-04T12:18:25.240402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/1656435333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1656435333},{"type":"v1","addr":"172.21.15.111:6835","nonce":1656435333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-05-04T12:18:31.997 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 7 2024-05-04T12:18:31.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.998+0000 7f369421b640 1 -- 172.21.15.52:0/873646240 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f36700678c0 msgr2=0x7f3670069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:31.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.998+0000 7f369421b640 1 --2- 172.21.15.52:0/873646240 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f36700678c0 0x7f3670069d80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f367c0b1000 tx=0x7f367c008040 comp rx=0 tx=0).stop 2024-05-04T12:18:31.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.998+0000 7f369421b640 1 -- 172.21.15.52:0/873646240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f368c06db40 msgr2=0x7f368c14b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.998+0000 7f369421b640 1 --2- 172.21.15.52:0/873646240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f368c06db40 0x7f368c14b740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f368800b440 tx=0x7f368800b900 comp rx=0 tx=0).stop 2024-05-04T12:18:32.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.998+0000 7f369421b640 1 -- 172.21.15.52:0/873646240 shutdown_connections 2024-05-04T12:18:32.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.998+0000 7f369421b640 1 --2- 172.21.15.52:0/873646240 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f36700678c0 0x7f3670069d80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.998+0000 7f369421b640 1 --2- 172.21.15.52:0/873646240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f368c15ae50 0x7f368c139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.998+0000 7f369421b640 1 --2- 172.21.15.52:0/873646240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f368c06f290 0x7f368c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.998+0000 7f369421b640 1 --2- 172.21.15.52:0/873646240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f368c06db40 0x7f368c14b740 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.998+0000 7f369421b640 1 -- 172.21.15.52:0/873646240 >> 172.21.15.52:0/873646240 conn(0x7f368c0b3190 msgr2=0x7f368c0b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:32.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.998+0000 7f369421b640 1 -- 172.21.15.52:0/873646240 shutdown_connections 2024-05-04T12:18:32.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:31.998+0000 7f369421b640 1 -- 172.21.15.52:0/873646240 wait complete. 2024-05-04T12:18:32.017 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:32.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a798d640 1 Processor -- start 2024-05-04T12:18:32.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a798d640 1 -- start start 2024-05-04T12:18:32.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a798d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f19a0141120 0x7f19a00aef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a798d640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19a00b0420 con 0x7f19a0162620 2024-05-04T12:18:32.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a798d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19a00af930 con 0x7f19a00af490 2024-05-04T12:18:32.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a798d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f19a00afc10 con 0x7f19a0141120 2024-05-04T12:18:32.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a5f2a640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f19a0162620 0x7f19a006e590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51644/0 (socket says 172.21.15.52:51644) 2024-05-04T12:18:32.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a5f2a640 1 -- 172.21.15.52:0/1523790909 learned_addr learned my addr 172.21.15.52:0/1523790909 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:32.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a5729640 1 --2- 172.21.15.52:0/1523790909 >> v2:172.21.15.111:3300/0 conn(0x7f19a0141120 0x7f19a00aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 -- 172.21.15.52:0/1523790909 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3765877403 0 0) 0x7f19a00b0420 con 0x7f19a0162620 2024-05-04T12:18:32.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 -- 172.21.15.52:0/1523790909 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f198c003530 con 0x7f19a0162620 2024-05-04T12:18:32.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 -- 172.21.15.52:0/1523790909 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2671058052 0 0) 0x7f198c003530 con 0x7f19a0162620 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 -- 172.21.15.52:0/1523790909 >> v2:172.21.15.111:3300/0 conn(0x7f19a0141120 msgr2=0x7f19a00aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 --2- 172.21.15.52:0/1523790909 >> v2:172.21.15.111:3300/0 conn(0x7f19a0141120 0x7f19a00aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 -- 172.21.15.52:0/1523790909 >> v1:172.21.15.111:6789/0 conn(0x7f19a00af490 legacy=0x7f19a0161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 -- 172.21.15.52:0/1523790909 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a00afef0 con 0x7f19a0162620 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 -- 172.21.15.52:0/1523790909 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f1998001a00 con 0x7f19a0162620 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 -- 172.21.15.52:0/1523790909 >> v1:172.21.15.52:6789/0 conn(0x7f19a0162620 legacy=0x7f19a006e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 --2- 172.21.15.52:0/1523790909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c0039c0 0x7f198c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 --2- 172.21.15.52:0/1523790909 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f198c004810 0x7f198c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 --2- 172.21.15.52:0/1523790909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f198c0051d0 0x7f198c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 -- 172.21.15.52:0/1523790909 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f19a00afc10 con 0x7f198c0039c0 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 -- 172.21.15.52:0/1523790909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f198c00c070 con 0x7f198c0051d0 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f1997fff640 1 -- 172.21.15.52:0/1523790909 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f198c00c270 con 0x7f198c004810 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a5729640 1 --2- 172.21.15.52:0/1523790909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c0039c0 0x7f198c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a5f2a640 1 --2- 172.21.15.52:0/1523790909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f198c0051d0 0x7f198c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a4f28640 1 --2- 172.21.15.52:0/1523790909 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f198c004810 0x7f198c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a5729640 1 -- 172.21.15.52:0/1523790909 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f198c004810 msgr2=0x7f198c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:32.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a5729640 1 --2- 172.21.15.52:0/1523790909 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f198c004810 0x7f198c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a5729640 1 -- 172.21.15.52:0/1523790909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f198c0051d0 msgr2=0x7f198c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:32.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a5729640 1 --2- 172.21.15.52:0/1523790909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f198c0051d0 0x7f198c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a5729640 1 -- 172.21.15.52:0/1523790909 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c003530 con 0x7f198c0039c0 2024-05-04T12:18:32.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a4f28640 1 --2- 172.21.15.52:0/1523790909 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f198c004810 0x7f198c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:32.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a5f2a640 1 --2- 172.21.15.52:0/1523790909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f198c0051d0 0x7f198c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:32.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.142+0000 7f19a5f2a640 1 -- 172.21.15.52:0/1523790909 reap_dead start 2024-05-04T12:18:32.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a5729640 1 --2- 172.21.15.52:0/1523790909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c0039c0 0x7f198c003de0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f199c009ae0 tx=0x7f199c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1346aca6bdb3e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:32.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f1997fff640 1 -- 172.21.15.52:0/1523790909 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c0b2070 con 0x7f198c0039c0 2024-05-04T12:18:32.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f1997fff640 1 -- 172.21.15.52:0/1523790909 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f199c0a5630 con 0x7f198c0039c0 2024-05-04T12:18:32.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f1997fff640 1 -- 172.21.15.52:0/1523790909 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c0096b0 con 0x7f198c0039c0 2024-05-04T12:18:32.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 -- 172.21.15.52:0/1523790909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c0039c0 msgr2=0x7f198c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 --2- 172.21.15.52:0/1523790909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c0039c0 0x7f198c003de0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f199c009ae0 tx=0x7f199c01ca40 comp rx=0 tx=0).stop 2024-05-04T12:18:32.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 -- 172.21.15.52:0/1523790909 shutdown_connections 2024-05-04T12:18:32.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 --2- 172.21.15.52:0/1523790909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c0039c0 0x7f198c003de0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 -- 172.21.15.52:0/1523790909 >> 172.21.15.52:0/1523790909 conn(0x7f19a0067070 msgr2=0x7f19a0067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:32.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 -- 172.21.15.52:0/1523790909 shutdown_connections 2024-05-04T12:18:32.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 -- 172.21.15.52:0/1523790909 wait complete. 2024-05-04T12:18:32.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 Processor -- start 2024-05-04T12:18:32.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 -- start start 2024-05-04T12:18:32.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c0039c0 0x7f19a013a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19a01e9f90 0x7f19a01ea410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19a0160950 0x7f19a0160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f19a0141680 con 0x7f198c0039c0 2024-05-04T12:18:32.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f19a0141500 con 0x7f19a0160950 2024-05-04T12:18:32.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a798d640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f19a0162c20 con 0x7f19a01e9f90 2024-05-04T12:18:32.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a5729640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c0039c0 0x7f19a013a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a5729640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c0039c0 0x7f19a013a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53768/0 (socket says 172.21.15.52:53768) 2024-05-04T12:18:32.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.146+0000 7f19a5729640 1 -- 172.21.15.52:0/470676073 learned_addr learned my addr 172.21.15.52:0/470676073 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:32.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f19a5729640 1 -- 172.21.15.52:0/470676073 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19a01e9f90 msgr2=0x7f19a01ea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f19a5729640 1 --2- 172.21.15.52:0/470676073 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19a01e9f90 0x7f19a01ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f19a5f2a640 1 --2- 172.21.15.52:0/470676073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19a0160950 0x7f19a0160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f19a5729640 1 -- 172.21.15.52:0/470676073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19a0160950 msgr2=0x7f19a0160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f19a5729640 1 --2- 172.21.15.52:0/470676073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19a0160950 0x7f19a0160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f19a5729640 1 -- 172.21.15.52:0/470676073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a01eaa50 con 0x7f198c0039c0 2024-05-04T12:18:32.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f19a5f2a640 1 --2- 172.21.15.52:0/470676073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19a0160950 0x7f19a0160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:32.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f19a5729640 1 --2- 172.21.15.52:0/470676073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c0039c0 0x7f19a013a7d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f199c0b0040 tx=0x7f199c0197e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:32.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f1995ffb640 1 -- 172.21.15.52:0/470676073 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c0b2040 con 0x7f198c0039c0 2024-05-04T12:18:32.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f1995ffb640 1 -- 172.21.15.52:0/470676073 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f199c00f000 con 0x7f198c0039c0 2024-05-04T12:18:32.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f1995ffb640 1 -- 172.21.15.52:0/470676073 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c004260 con 0x7f198c0039c0 2024-05-04T12:18:32.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f19a798d640 1 -- 172.21.15.52:0/470676073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a01ead30 con 0x7f198c0039c0 2024-05-04T12:18:32.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f19a798d640 1 -- 172.21.15.52:0/470676073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a01eb2f0 con 0x7f198c0039c0 2024-05-04T12:18:32.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f19a798d640 1 -- 172.21.15.52:0/470676073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a0062e50 con 0x7f198c0039c0 2024-05-04T12:18:32.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f1995ffb640 1 -- 172.21.15.52:0/470676073 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f199c003620 con 0x7f198c0039c0 2024-05-04T12:18:32.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f1995ffb640 1 --2- 172.21.15.52:0/470676073 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19840678c0 0x7f1984069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f1995ffb640 1 -- 172.21.15.52:0/470676073 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f199c11f130 con 0x7f198c0039c0 2024-05-04T12:18:32.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f19a4f28640 1 --2- 172.21.15.52:0/470676073 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19840678c0 0x7f1984069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.150+0000 7f19a4f28640 1 --2- 172.21.15.52:0/470676073 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19840678c0 0x7f1984069d80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f19a0161e20 tx=0x7f1990009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:32.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.154+0000 7f1995ffb640 1 -- 172.21.15.52:0/470676073 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f199c016780 con 0x7f198c0039c0 2024-05-04T12:18:32.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.318+0000 7f19a798d640 1 -- 172.21.15.52:0/470676073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19a0075320 con 0x7f198c0039c0 2024-05-04T12:18:32.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.318+0000 7f1995ffb640 1 -- 172.21.15.52:0/470676073 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f199c0efbe0 con 0x7f198c0039c0 2024-05-04T12:18:32.321 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:32.321 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:32.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.318+0000 7f19a798d640 1 -- 172.21.15.52:0/470676073 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19840678c0 msgr2=0x7f1984069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.318+0000 7f19a798d640 1 --2- 172.21.15.52:0/470676073 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19840678c0 0x7f1984069d80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f19a0161e20 tx=0x7f1990009040 comp rx=0 tx=0).stop 2024-05-04T12:18:32.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.322+0000 7f19a798d640 1 -- 172.21.15.52:0/470676073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c0039c0 msgr2=0x7f19a013a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.322+0000 7f19a798d640 1 --2- 172.21.15.52:0/470676073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c0039c0 0x7f19a013a7d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f199c0b0040 tx=0x7f199c0197e0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.322+0000 7f19a798d640 1 -- 172.21.15.52:0/470676073 shutdown_connections 2024-05-04T12:18:32.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.322+0000 7f19a798d640 1 --2- 172.21.15.52:0/470676073 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19840678c0 0x7f1984069d80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.322+0000 7f19a798d640 1 --2- 172.21.15.52:0/470676073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19a0160950 0x7f19a0160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.322+0000 7f19a798d640 1 --2- 172.21.15.52:0/470676073 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19a01e9f90 0x7f19a01ea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.322+0000 7f19a798d640 1 --2- 172.21.15.52:0/470676073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c0039c0 0x7f19a013a7d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.322+0000 7f19a798d640 1 -- 172.21.15.52:0/470676073 >> 172.21.15.52:0/470676073 conn(0x7f19a0067070 msgr2=0x7f19a015fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:32.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.322+0000 7f19a798d640 1 -- 172.21.15.52:0/470676073 shutdown_connections 2024-05-04T12:18:32.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.322+0000 7f19a798d640 1 -- 172.21.15.52:0/470676073 wait complete. 2024-05-04T12:18:32.339 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:18:32.339 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:18:32.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff33e1bf640 1 Processor -- start 2024-05-04T12:18:32.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff33e1bf640 1 -- start start 2024-05-04T12:18:32.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff33e1bf640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff338070de0 0x7ff33815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff33e1bf640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff338161cb0 0x7ff338160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff33e1bf640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff33815ec80 con 0x7ff3380d08a0 2024-05-04T12:18:32.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff33e1bf640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ff33815f8b0 con 0x7ff338161cb0 2024-05-04T12:18:32.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff33e1bf640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7ff33815ef60 con 0x7ff338070de0 2024-05-04T12:18:32.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff337fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff338161cb0 0x7ff338160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff337fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff338161cb0 0x7ff338160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53772/0 (socket says 172.21.15.52:53772) 2024-05-04T12:18:32.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff337fff640 1 -- 172.21.15.52:0/3663838525 learned_addr learned my addr 172.21.15.52:0/3663838525 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:32.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff336ffd640 1 --2- 172.21.15.52:0/3663838525 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff338070de0 0x7ff33815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff337fff640 1 -- 172.21.15.52:0/3663838525 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff338070de0 msgr2=0x7ff33815fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:32.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff337fff640 1 --2- 172.21.15.52:0/3663838525 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff338070de0 0x7ff33815fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff337fff640 1 -- 172.21.15.52:0/3663838525 >> v1:172.21.15.111:6789/0 conn(0x7ff3380d08a0 legacy=0x7ff3380706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.454+0000 7ff337fff640 1 -- 172.21.15.52:0/3663838525 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33815f240 con 0x7ff338161cb0 2024-05-04T12:18:32.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff337fff640 1 --2- 172.21.15.52:0/3663838525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff338161cb0 0x7ff338160280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff330009870 tx=0x7ff33001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533d6c12fe105279 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:32.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff3367fc640 1 -- 172.21.15.52:0/3663838525 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3300b1070 con 0x7ff338161cb0 2024-05-04T12:18:32.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff3367fc640 1 -- 172.21.15.52:0/3663838525 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3300a8050 con 0x7ff338161cb0 2024-05-04T12:18:32.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff3367fc640 1 -- 172.21.15.52:0/3663838525 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3300ac860 con 0x7ff338161cb0 2024-05-04T12:18:32.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 -- 172.21.15.52:0/3663838525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff338161cb0 msgr2=0x7ff338160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 --2- 172.21.15.52:0/3663838525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff338161cb0 0x7ff338160280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff330009870 tx=0x7ff33001c4d0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 -- 172.21.15.52:0/3663838525 shutdown_connections 2024-05-04T12:18:32.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 --2- 172.21.15.52:0/3663838525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff338161cb0 0x7ff338160280 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 --2- 172.21.15.52:0/3663838525 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff338070de0 0x7ff33815fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 -- 172.21.15.52:0/3663838525 >> 172.21.15.52:0/3663838525 conn(0x7ff3380d12c0 msgr2=0x7ff3380d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:32.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 -- 172.21.15.52:0/3663838525 shutdown_connections 2024-05-04T12:18:32.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 -- 172.21.15.52:0/3663838525 wait complete. 2024-05-04T12:18:32.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 Processor -- start 2024-05-04T12:18:32.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 -- start start 2024-05-04T12:18:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff338070de0 0x7ff33814d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3380d08a0 0x7ff3381580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff338161cb0 0x7ff3381585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff33816afc0 con 0x7ff338070de0 2024-05-04T12:18:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff33816ae40 con 0x7ff3380d08a0 2024-05-04T12:18:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff33e1bf640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7ff33816b140 con 0x7ff338161cb0 2024-05-04T12:18:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff3377fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff338070de0 0x7ff33814d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff3377fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff338070de0 0x7ff33814d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53788/0 (socket says 172.21.15.52:53788) 2024-05-04T12:18:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff3377fe640 1 -- 172.21.15.52:0/1234214248 learned_addr learned my addr 172.21.15.52:0/1234214248 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff337fff640 1 --2- 172.21.15.52:0/1234214248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff338161cb0 0x7ff3381585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff336ffd640 1 --2- 172.21.15.52:0/1234214248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3380d08a0 0x7ff3381580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff3377fe640 1 -- 172.21.15.52:0/1234214248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff338161cb0 msgr2=0x7ff3381585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff3377fe640 1 --2- 172.21.15.52:0/1234214248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff338161cb0 0x7ff3381585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff3377fe640 1 -- 172.21.15.52:0/1234214248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3380d08a0 msgr2=0x7ff3381580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff3377fe640 1 --2- 172.21.15.52:0/1234214248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3380d08a0 0x7ff3381580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.458+0000 7ff3377fe640 1 -- 172.21.15.52:0/1234214248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33814ab90 con 0x7ff338070de0 2024-05-04T12:18:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.462+0000 7ff3377fe640 1 --2- 172.21.15.52:0/1234214248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff338070de0 0x7ff33814d920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff32000c420 tx=0x7ff32000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.462+0000 7ff32ffff640 1 -- 172.21.15.52:0/1234214248 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3200052d0 con 0x7ff338070de0 2024-05-04T12:18:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.462+0000 7ff32ffff640 1 -- 172.21.15.52:0/1234214248 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff320005470 con 0x7ff338070de0 2024-05-04T12:18:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.462+0000 7ff33e1bf640 1 -- 172.21.15.52:0/1234214248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33814ae70 con 0x7ff338070de0 2024-05-04T12:18:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.462+0000 7ff32ffff640 1 -- 172.21.15.52:0/1234214248 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320007260 con 0x7ff338070de0 2024-05-04T12:18:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.462+0000 7ff33e1bf640 1 -- 172.21.15.52:0/1234214248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33814b430 con 0x7ff338070de0 2024-05-04T12:18:32.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.462+0000 7ff33e1bf640 1 -- 172.21.15.52:0/1234214248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff338062e50 con 0x7ff338070de0 2024-05-04T12:18:32.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.462+0000 7ff32ffff640 1 -- 172.21.15.52:0/1234214248 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff320007400 con 0x7ff338070de0 2024-05-04T12:18:32.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.462+0000 7ff32ffff640 1 --2- 172.21.15.52:0/1234214248 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff3180678c0 0x7ff318069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.462+0000 7ff32ffff640 1 -- 172.21.15.52:0/1234214248 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff320085120 con 0x7ff338070de0 2024-05-04T12:18:32.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.462+0000 7ff336ffd640 1 --2- 172.21.15.52:0/1234214248 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff3180678c0 0x7ff318069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.462+0000 7ff336ffd640 1 --2- 172.21.15.52:0/1234214248 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff3180678c0 0x7ff318069d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff3381fa900 tx=0x7ff32800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:32.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.466+0000 7ff32ffff640 1 -- 172.21.15.52:0/1234214248 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff320055610 con 0x7ff338070de0 2024-05-04T12:18:32.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.626+0000 7ff33e1bf640 1 -- 172.21.15.52:0/1234214248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff33814f020 con 0x7ff338070de0 2024-05-04T12:18:32.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.626+0000 7ff32ffff640 1 -- 172.21.15.52:0/1234214248 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7ff320059910 con 0x7ff338070de0 2024-05-04T12:18:32.630 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:18:32.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.630+0000 7ff33e1bf640 1 -- 172.21.15.52:0/1234214248 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff3180678c0 msgr2=0x7ff318069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.630+0000 7ff33e1bf640 1 --2- 172.21.15.52:0/1234214248 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff3180678c0 0x7ff318069d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff3381fa900 tx=0x7ff32800b040 comp rx=0 tx=0).stop 2024-05-04T12:18:32.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.630+0000 7ff33e1bf640 1 -- 172.21.15.52:0/1234214248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff338070de0 msgr2=0x7ff33814d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.630+0000 7ff33e1bf640 1 --2- 172.21.15.52:0/1234214248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff338070de0 0x7ff33814d920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff32000c420 tx=0x7ff32000c8e0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.630+0000 7ff33e1bf640 1 -- 172.21.15.52:0/1234214248 shutdown_connections 2024-05-04T12:18:32.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.630+0000 7ff33e1bf640 1 --2- 172.21.15.52:0/1234214248 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff3180678c0 0x7ff318069d80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.630+0000 7ff33e1bf640 1 --2- 172.21.15.52:0/1234214248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff338161cb0 0x7ff3381585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.630+0000 7ff33e1bf640 1 --2- 172.21.15.52:0/1234214248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3380d08a0 0x7ff3381580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.630+0000 7ff33e1bf640 1 --2- 172.21.15.52:0/1234214248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff338070de0 0x7ff33814d920 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.630+0000 7ff33e1bf640 1 -- 172.21.15.52:0/1234214248 >> 172.21.15.52:0/1234214248 conn(0x7ff3380d12c0 msgr2=0x7ff338154d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:32.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.630+0000 7ff33e1bf640 1 -- 172.21.15.52:0/1234214248 shutdown_connections 2024-05-04T12:18:32.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.630+0000 7ff33e1bf640 1 -- 172.21.15.52:0/1234214248 wait complete. 2024-05-04T12:18:32.649 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:18:32.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.762+0000 7f366bc32640 1 Processor -- start 2024-05-04T12:18:32.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.762+0000 7f366bc32640 1 -- start start 2024-05-04T12:18:32.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f366bc32640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36641626c0 0x7f3664071f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f366bc32640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36640cd210 con 0x7f36640cc990 2024-05-04T12:18:32.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f366bc32640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36640cd4f0 con 0x7f36640aedf0 2024-05-04T12:18:32.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f366bc32640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f366406df30 con 0x7f36641626c0 2024-05-04T12:18:32.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36691cd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f36640cc990 0x7f36640ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51656/0 (socket says 172.21.15.52:51656) 2024-05-04T12:18:32.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36691cd640 1 -- 172.21.15.52:0/3179134500 learned_addr learned my addr 172.21.15.52:0/3179134500 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:32.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36699ce640 1 --2- 172.21.15.52:0/3179134500 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36641626c0 0x7f3664071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 -- 172.21.15.52:0/3179134500 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2260908213 0 0) 0x7f36640cd210 con 0x7f36640cc990 2024-05-04T12:18:32.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 -- 172.21.15.52:0/3179134500 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3650003530 con 0x7f36640cc990 2024-05-04T12:18:32.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 -- 172.21.15.52:0/3179134500 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2879574799 0 0) 0x7f3650003530 con 0x7f36640cc990 2024-05-04T12:18:32.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 -- 172.21.15.52:0/3179134500 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36641626c0 msgr2=0x7f3664071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:32.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 --2- 172.21.15.52:0/3179134500 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36641626c0 0x7f3664071f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 -- 172.21.15.52:0/3179134500 >> v1:172.21.15.111:6789/0 conn(0x7f36640aedf0 legacy=0x7f366406e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 -- 172.21.15.52:0/3179134500 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366406e210 con 0x7f36640cc990 2024-05-04T12:18:32.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 -- 172.21.15.52:0/3179134500 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f3654001680 con 0x7f36640cc990 2024-05-04T12:18:32.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 -- 172.21.15.52:0/3179134500 >> v1:172.21.15.52:6789/0 conn(0x7f36640cc990 legacy=0x7f36640ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 --2- 172.21.15.52:0/3179134500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36500039f0 0x7f3650003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 --2- 172.21.15.52:0/3179134500 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36641626c0 0x7f3650004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 --2- 172.21.15.52:0/3179134500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3650004d50 0x7f36500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 -- 172.21.15.52:0/3179134500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f366406df30 con 0x7f3650004d50 2024-05-04T12:18:32.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 -- 172.21.15.52:0/3179134500 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f365000b9d0 con 0x7f36500039f0 2024-05-04T12:18:32.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36689cc640 1 -- 172.21.15.52:0/3179134500 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f365000bbd0 con 0x7f36641626c0 2024-05-04T12:18:32.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f366a1cf640 1 --2- 172.21.15.52:0/3179134500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3650004d50 0x7f36500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36699ce640 1 --2- 172.21.15.52:0/3179134500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36500039f0 0x7f3650003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f36691cd640 1 --2- 172.21.15.52:0/3179134500 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36641626c0 0x7f3650004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f366a1cf640 1 -- 172.21.15.52:0/3179134500 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36641626c0 msgr2=0x7f3650004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:32.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f366a1cf640 1 --2- 172.21.15.52:0/3179134500 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36641626c0 0x7f3650004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f366a1cf640 1 -- 172.21.15.52:0/3179134500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36500039f0 msgr2=0x7f3650003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:32.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f366a1cf640 1 --2- 172.21.15.52:0/3179134500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36500039f0 0x7f3650003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.766+0000 7f366a1cf640 1 -- 172.21.15.52:0/3179134500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3650003530 con 0x7f3650004d50 2024-05-04T12:18:32.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366a1cf640 1 --2- 172.21.15.52:0/3179134500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3650004d50 0x7f36500051d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f365c001f60 tx=0x7f365c01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f4a98a897b8ed8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:32.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f36689cc640 1 -- 172.21.15.52:0/3179134500 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c0ac070 con 0x7f3650004d50 2024-05-04T12:18:32.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f36689cc640 1 -- 172.21.15.52:0/3179134500 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365c0a6670 con 0x7f3650004d50 2024-05-04T12:18:32.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f36689cc640 1 -- 172.21.15.52:0/3179134500 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c001b60 con 0x7f3650004d50 2024-05-04T12:18:32.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 -- 172.21.15.52:0/3179134500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3650004d50 msgr2=0x7f36500051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 --2- 172.21.15.52:0/3179134500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3650004d50 0x7f36500051d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f365c001f60 tx=0x7f365c01c880 comp rx=0 tx=0).stop 2024-05-04T12:18:32.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366a1cf640 1 -- 172.21.15.52:0/3179134500 reap_dead start 2024-05-04T12:18:32.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 -- 172.21.15.52:0/3179134500 shutdown_connections 2024-05-04T12:18:32.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 -- 172.21.15.52:0/3179134500 >> 172.21.15.52:0/3179134500 conn(0x7f3664067730 msgr2=0x7f36640b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:32.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 -- 172.21.15.52:0/3179134500 shutdown_connections 2024-05-04T12:18:32.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 -- 172.21.15.52:0/3179134500 wait complete. 2024-05-04T12:18:32.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 Processor -- start 2024-05-04T12:18:32.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 -- start start 2024-05-04T12:18:32.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36641402b0 0x7f3664142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3664142ea0 0x7f3664141600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3664141b40 0x7f3664141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f36640ccf70 con 0x7f3664142ea0 2024-05-04T12:18:32.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f36640ccdf0 con 0x7f36641402b0 2024-05-04T12:18:32.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f3664071f50 con 0x7f3664141b40 2024-05-04T12:18:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f36691cd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3664142ea0 0x7f3664141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f36691cd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3664142ea0 0x7f3664141600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53814/0 (socket says 172.21.15.52:53814) 2024-05-04T12:18:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f36691cd640 1 -- 172.21.15.52:0/4155830388 learned_addr learned my addr 172.21.15.52:0/4155830388 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366a1cf640 1 --2- 172.21.15.52:0/4155830388 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3664141b40 0x7f3664141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f36699ce640 1 --2- 172.21.15.52:0/4155830388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36641402b0 0x7f3664142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f36691cd640 1 -- 172.21.15.52:0/4155830388 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3664141b40 msgr2=0x7f3664141fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f36691cd640 1 --2- 172.21.15.52:0/4155830388 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3664141b40 0x7f3664141fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f36691cd640 1 -- 172.21.15.52:0/4155830388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36641402b0 msgr2=0x7f3664142960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f36691cd640 1 --2- 172.21.15.52:0/4155830388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36641402b0 0x7f3664142960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f36691cd640 1 -- 172.21.15.52:0/4155830388 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3664153f10 con 0x7f3664142ea0 2024-05-04T12:18:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f36691cd640 1 --2- 172.21.15.52:0/4155830388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3664142ea0 0x7f3664141600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f36540119b0 tx=0x7f3654011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f365a7fc640 1 -- 172.21.15.52:0/4155830388 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654010040 con 0x7f3664142ea0 2024-05-04T12:18:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f365a7fc640 1 -- 172.21.15.52:0/4155830388 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365401a070 con 0x7f3664142ea0 2024-05-04T12:18:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 -- 172.21.15.52:0/4155830388 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3664140b20 con 0x7f3664142ea0 2024-05-04T12:18:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f365a7fc640 1 -- 172.21.15.52:0/4155830388 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365400fe40 con 0x7f3664142ea0 2024-05-04T12:18:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.770+0000 7f366bc32640 1 -- 172.21.15.52:0/4155830388 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36641410e0 con 0x7f3664142ea0 2024-05-04T12:18:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.774+0000 7f366bc32640 1 -- 172.21.15.52:0/4155830388 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3664062e50 con 0x7f3664142ea0 2024-05-04T12:18:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.774+0000 7f365a7fc640 1 -- 172.21.15.52:0/4155830388 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f365401f460 con 0x7f3664142ea0 2024-05-04T12:18:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.774+0000 7f365a7fc640 1 --2- 172.21.15.52:0/4155830388 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f36480678c0 0x7f3648069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.774+0000 7f365a7fc640 1 -- 172.21.15.52:0/4155830388 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3654088f20 con 0x7f3664142ea0 2024-05-04T12:18:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.774+0000 7f36699ce640 1 --2- 172.21.15.52:0/4155830388 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f36480678c0 0x7f3648069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:32.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.774+0000 7f36699ce640 1 --2- 172.21.15.52:0/4155830388 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f36480678c0 0x7f3648069d80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3660009a30 tx=0x7f3660008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:32.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.778+0000 7f365a7fc640 1 -- 172.21.15.52:0/4155830388 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3654059bb0 con 0x7f3664142ea0 2024-05-04T12:18:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.938+0000 7f366bc32640 1 -- 172.21.15.52:0/4155830388 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3664075320 con 0x7f3664142ea0 2024-05-04T12:18:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.938+0000 7f365a7fc640 1 -- 172.21.15.52:0/4155830388 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f36540594b0 con 0x7f3664142ea0 2024-05-04T12:18:32.940 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:18:32.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.942+0000 7f366bc32640 1 -- 172.21.15.52:0/4155830388 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f36480678c0 msgr2=0x7f3648069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.942+0000 7f366bc32640 1 --2- 172.21.15.52:0/4155830388 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f36480678c0 0x7f3648069d80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3660009a30 tx=0x7f3660008040 comp rx=0 tx=0).stop 2024-05-04T12:18:32.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.942+0000 7f366bc32640 1 -- 172.21.15.52:0/4155830388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3664142ea0 msgr2=0x7f3664141600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:32.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.942+0000 7f366bc32640 1 --2- 172.21.15.52:0/4155830388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3664142ea0 0x7f3664141600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f36540119b0 tx=0x7f3654011e70 comp rx=0 tx=0).stop 2024-05-04T12:18:32.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.942+0000 7f366bc32640 1 -- 172.21.15.52:0/4155830388 shutdown_connections 2024-05-04T12:18:32.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.942+0000 7f366bc32640 1 --2- 172.21.15.52:0/4155830388 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f36480678c0 0x7f3648069d80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.942+0000 7f366bc32640 1 --2- 172.21.15.52:0/4155830388 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3664141b40 0x7f3664141fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.942+0000 7f366bc32640 1 --2- 172.21.15.52:0/4155830388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3664142ea0 0x7f3664141600 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.942+0000 7f366bc32640 1 --2- 172.21.15.52:0/4155830388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36641402b0 0x7f3664142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.942+0000 7f366bc32640 1 -- 172.21.15.52:0/4155830388 >> 172.21.15.52:0/4155830388 conn(0x7f3664067730 msgr2=0x7f36640cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.942+0000 7f366bc32640 1 -- 172.21.15.52:0/4155830388 shutdown_connections 2024-05-04T12:18:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:32.942+0000 7f366bc32640 1 -- 172.21.15.52:0/4155830388 wait complete. 2024-05-04T12:18:32.960 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:18:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.074+0000 7f066f8e3640 1 Processor -- start 2024-05-04T12:18:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.074+0000 7f066f8e3640 1 -- start start 2024-05-04T12:18:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f066f8e3640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0668145960 0x7f0668160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f066f8e3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f06680ccea0 0x7f066806e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f066f8e3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f066806dbf0 con 0x7f066815c590 2024-05-04T12:18:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f066f8e3640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f0668160ad0 con 0x7f0668145960 2024-05-04T12:18:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f066f8e3640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f066806cfe0 con 0x7f06680ccea0 2024-05-04T12:18:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f066d67f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0668145960 0x7f0668160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f066d67f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0668145960 0x7f0668160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53824/0 (socket says 172.21.15.52:53824) 2024-05-04T12:18:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f066d67f640 1 -- 172.21.15.52:0/2621493940 learned_addr learned my addr 172.21.15.52:0/2621493940 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f066d67f640 1 -- 172.21.15.52:0/2621493940 >> v2:172.21.15.111:3300/0 conn(0x7f06680ccea0 msgr2=0x7f066806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f066d67f640 1 --2- 172.21.15.52:0/2621493940 >> v2:172.21.15.111:3300/0 conn(0x7f06680ccea0 0x7f066806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f066d67f640 1 -- 172.21.15.52:0/2621493940 >> v1:172.21.15.111:6789/0 conn(0x7f066815c590 legacy=0x7f06680cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f066d67f640 1 -- 172.21.15.52:0/2621493940 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066806d2c0 con 0x7f0668145960 2024-05-04T12:18:33.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f066d67f640 1 --2- 172.21.15.52:0/2621493940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0668145960 0x7f0668160590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f06680737e0 tx=0x7f066401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7da30caac0d79fe server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f065ffff640 1 -- 172.21.15.52:0/2621493940 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06640ab620 con 0x7f0668145960 2024-05-04T12:18:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f065ffff640 1 -- 172.21.15.52:0/2621493940 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06640b0070 con 0x7f0668145960 2024-05-04T12:18:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.078+0000 7f065ffff640 1 -- 172.21.15.52:0/2621493940 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06640ab9e0 con 0x7f0668145960 2024-05-04T12:18:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 -- 172.21.15.52:0/2621493940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0668145960 msgr2=0x7f0668160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 --2- 172.21.15.52:0/2621493940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0668145960 0x7f0668160590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f06680737e0 tx=0x7f066401c420 comp rx=0 tx=0).stop 2024-05-04T12:18:33.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 -- 172.21.15.52:0/2621493940 shutdown_connections 2024-05-04T12:18:33.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 --2- 172.21.15.52:0/2621493940 >> v2:172.21.15.111:3300/0 conn(0x7f06680ccea0 0x7f066806e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 --2- 172.21.15.52:0/2621493940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0668145960 0x7f0668160590 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 -- 172.21.15.52:0/2621493940 >> 172.21.15.52:0/2621493940 conn(0x7f0668067730 msgr2=0x7f06680b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:33.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 -- 172.21.15.52:0/2621493940 shutdown_connections 2024-05-04T12:18:33.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 -- 172.21.15.52:0/2621493940 wait complete. 2024-05-04T12:18:33.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 Processor -- start 2024-05-04T12:18:33.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 -- start start 2024-05-04T12:18:33.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06680ccea0 0x7f06680b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0668145960 0x7f066815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f066815c590 0x7f066815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f066806cdc0 con 0x7f06680ccea0 2024-05-04T12:18:33.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f066806cc40 con 0x7f066815c590 2024-05-04T12:18:33.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f0668145d40 con 0x7f0668145960 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066d67f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06680ccea0 0x7f06680b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066d67f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06680ccea0 0x7f06680b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53832/0 (socket says 172.21.15.52:53832) 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066d67f640 1 -- 172.21.15.52:0/1931041218 learned_addr learned my addr 172.21.15.52:0/1931041218 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066ce7e640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0668145960 0x7f066815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066de80640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f066815c590 0x7f066815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066d67f640 1 -- 172.21.15.52:0/1931041218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0668145960 msgr2=0x7f066815a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066d67f640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0668145960 0x7f066815a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066d67f640 1 -- 172.21.15.52:0/1931041218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f066815c590 msgr2=0x7f066815abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066d67f640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f066815c590 0x7f066815abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066d67f640 1 -- 172.21.15.52:0/1931041218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066815b540 con 0x7f06680ccea0 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066de80640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f066815c590 0x7f066815abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066d67f640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06680ccea0 0x7f06680b06c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f066401ccc0 tx=0x7f066401ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f065dffb640 1 -- 172.21.15.52:0/1931041218 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06640abd70 con 0x7f06680ccea0 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f065dffb640 1 -- 172.21.15.52:0/1931041218 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06640be3f0 con 0x7f06680ccea0 2024-05-04T12:18:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f065dffb640 1 -- 172.21.15.52:0/1931041218 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06640be7e0 con 0x7f06680ccea0 2024-05-04T12:18:33.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 -- 172.21.15.52:0/1931041218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066815b260 con 0x7f06680ccea0 2024-05-04T12:18:33.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.082+0000 7f066f8e3640 1 -- 172.21.15.52:0/1931041218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06680ae760 con 0x7f06680ccea0 2024-05-04T12:18:33.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.086+0000 7f066f8e3640 1 -- 172.21.15.52:0/1931041218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0668062e50 con 0x7f06680ccea0 2024-05-04T12:18:33.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.086+0000 7f065dffb640 1 -- 172.21.15.52:0/1931041218 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f06640b43f0 con 0x7f06680ccea0 2024-05-04T12:18:33.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.086+0000 7f065dffb640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f064c0678c0 0x7f064c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.086+0000 7f065dffb640 1 -- 172.21.15.52:0/1931041218 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f066411fff0 con 0x7f06680ccea0 2024-05-04T12:18:33.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.086+0000 7f066ce7e640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f064c0678c0 0x7f064c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.086+0000 7f066ce7e640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f064c0678c0 0x7f064c069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06680aead0 tx=0x7f0658009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:33.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.090+0000 7f065dffb640 1 -- 172.21.15.52:0/1931041218 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f06640c4060 con 0x7f06680ccea0 2024-05-04T12:18:33.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.242+0000 7f066f8e3640 1 -- 172.21.15.52:0/1931041218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0668075320 con 0x7f06680ccea0 2024-05-04T12:18:33.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.242+0000 7f065dffb640 1 -- 172.21.15.52:0/1931041218 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f06640160d0 con 0x7f06680ccea0 2024-05-04T12:18:33.245 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:18:33.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.246+0000 7f066f8e3640 1 -- 172.21.15.52:0/1931041218 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f064c0678c0 msgr2=0x7f064c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.246+0000 7f066f8e3640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f064c0678c0 0x7f064c069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06680aead0 tx=0x7f0658009040 comp rx=0 tx=0).stop 2024-05-04T12:18:33.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.246+0000 7f066f8e3640 1 -- 172.21.15.52:0/1931041218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06680ccea0 msgr2=0x7f06680b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.246+0000 7f066f8e3640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06680ccea0 0x7f06680b06c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f066401ccc0 tx=0x7f066401ac60 comp rx=0 tx=0).stop 2024-05-04T12:18:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.246+0000 7f066f8e3640 1 -- 172.21.15.52:0/1931041218 shutdown_connections 2024-05-04T12:18:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.246+0000 7f066f8e3640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f064c0678c0 0x7f064c069d80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.246+0000 7f066f8e3640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f066815c590 0x7f066815abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.246+0000 7f066f8e3640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0668145960 0x7f066815a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.246+0000 7f066f8e3640 1 --2- 172.21.15.52:0/1931041218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06680ccea0 0x7f06680b06c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.246+0000 7f066f8e3640 1 -- 172.21.15.52:0/1931041218 >> 172.21.15.52:0/1931041218 conn(0x7f0668067730 msgr2=0x7f066813b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.246+0000 7f066f8e3640 1 -- 172.21.15.52:0/1931041218 shutdown_connections 2024-05-04T12:18:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.246+0000 7f066f8e3640 1 -- 172.21.15.52:0/1931041218 wait complete. 2024-05-04T12:18:33.263 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T12:18:33.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.378+0000 7f07b701e640 1 Processor -- start 2024-05-04T12:18:33.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.378+0000 7f07b701e640 1 -- start start 2024-05-04T12:18:33.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07b701e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f07b0073590 0x7f07b006def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07b701e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f07b006e430 0x7f07b013eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07b701e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b013e630 con 0x7f07b013dec0 2024-05-04T12:18:33.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07b701e640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f07b006ec90 con 0x7f07b0073590 2024-05-04T12:18:33.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07b701e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f07b006ef70 con 0x7f07b006e430 2024-05-04T12:18:33.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07affff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f07b0073590 0x7f07b006def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07affff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f07b0073590 0x7f07b006def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53836/0 (socket says 172.21.15.52:53836) 2024-05-04T12:18:33.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07affff640 1 -- 172.21.15.52:0/2771649174 learned_addr learned my addr 172.21.15.52:0/2771649174 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:33.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07b55bb640 1 --2- 172.21.15.52:0/2771649174 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f07b006e430 0x7f07b013eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07affff640 1 -- 172.21.15.52:0/2771649174 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f07b006e430 msgr2=0x7f07b013eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:33.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07affff640 1 --2- 172.21.15.52:0/2771649174 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f07b006e430 0x7f07b013eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07affff640 1 -- 172.21.15.52:0/2771649174 >> v1:172.21.15.111:6789/0 conn(0x7f07b013dec0 legacy=0x7f07b0072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07affff640 1 -- 172.21.15.52:0/2771649174 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b006f250 con 0x7f07b0073590 2024-05-04T12:18:33.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07affff640 1 --2- 172.21.15.52:0/2771649174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07b0073590 0x7f07b006def0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009870 tx=0x7f07a401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8821b13f1772d064 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:33.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07af7fe640 1 -- 172.21.15.52:0/2771649174 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40aa430 con 0x7f07b0073590 2024-05-04T12:18:33.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07af7fe640 1 -- 172.21.15.52:0/2771649174 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a40af070 con 0x7f07b0073590 2024-05-04T12:18:33.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07af7fe640 1 -- 172.21.15.52:0/2771649174 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40aa7f0 con 0x7f07b0073590 2024-05-04T12:18:33.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07b701e640 1 -- 172.21.15.52:0/2771649174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07b0073590 msgr2=0x7f07b006def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07b701e640 1 --2- 172.21.15.52:0/2771649174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07b0073590 0x7f07b006def0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009870 tx=0x7f07a401c530 comp rx=0 tx=0).stop 2024-05-04T12:18:33.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07b701e640 1 -- 172.21.15.52:0/2771649174 shutdown_connections 2024-05-04T12:18:33.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07b701e640 1 --2- 172.21.15.52:0/2771649174 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f07b006e430 0x7f07b013eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07b701e640 1 --2- 172.21.15.52:0/2771649174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07b0073590 0x7f07b006def0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07b701e640 1 -- 172.21.15.52:0/2771649174 >> 172.21.15.52:0/2771649174 conn(0x7f07b0067070 msgr2=0x7f07b0067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:33.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07b701e640 1 -- 172.21.15.52:0/2771649174 shutdown_connections 2024-05-04T12:18:33.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.382+0000 7f07b701e640 1 -- 172.21.15.52:0/2771649174 wait complete. 2024-05-04T12:18:33.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b701e640 1 Processor -- start 2024-05-04T12:18:33.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b701e640 1 -- start start 2024-05-04T12:18:33.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b701e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07b006e430 0x7f07b014f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b701e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f07b0073590 0x7f07b015a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b701e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07b013dec0 0x7f07b015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b701e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07b013f860 con 0x7f07b006e430 2024-05-04T12:18:33.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b701e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f07b006ea70 con 0x7f07b013dec0 2024-05-04T12:18:33.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b701e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f07b013f9e0 con 0x7f07b0073590 2024-05-04T12:18:33.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b4dba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07b006e430 0x7f07b014f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b4dba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07b006e430 0x7f07b014f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53852/0 (socket says 172.21.15.52:53852) 2024-05-04T12:18:33.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b4dba640 1 -- 172.21.15.52:0/90586359 learned_addr learned my addr 172.21.15.52:0/90586359 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:33.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07affff640 1 --2- 172.21.15.52:0/90586359 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f07b0073590 0x7f07b015a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b55bb640 1 --2- 172.21.15.52:0/90586359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07b013dec0 0x7f07b015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b4dba640 1 -- 172.21.15.52:0/90586359 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f07b0073590 msgr2=0x7f07b015a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b4dba640 1 --2- 172.21.15.52:0/90586359 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f07b0073590 0x7f07b015a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b4dba640 1 -- 172.21.15.52:0/90586359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07b013dec0 msgr2=0x7f07b015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b4dba640 1 --2- 172.21.15.52:0/90586359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07b013dec0 0x7f07b015a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b4dba640 1 -- 172.21.15.52:0/90586359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b013a0d0 con 0x7f07b006e430 2024-05-04T12:18:33.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b4dba640 1 --2- 172.21.15.52:0/90586359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07b006e430 0x7f07b014f850 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f07a000c370 tx=0x7f07a000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:33.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07ad7fa640 1 -- 172.21.15.52:0/90586359 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a00052d0 con 0x7f07b006e430 2024-05-04T12:18:33.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07ad7fa640 1 -- 172.21.15.52:0/90586359 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a0005470 con 0x7f07b006e430 2024-05-04T12:18:33.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07ad7fa640 1 -- 172.21.15.52:0/90586359 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0007260 con 0x7f07b006e430 2024-05-04T12:18:33.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b701e640 1 -- 172.21.15.52:0/90586359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b0134be0 con 0x7f07b006e430 2024-05-04T12:18:33.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b701e640 1 -- 172.21.15.52:0/90586359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b01351a0 con 0x7f07b006e430 2024-05-04T12:18:33.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.386+0000 7f07b701e640 1 -- 172.21.15.52:0/90586359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b0062e50 con 0x7f07b006e430 2024-05-04T12:18:33.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.390+0000 7f07ad7fa640 1 -- 172.21.15.52:0/90586359 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f07a0007400 con 0x7f07b006e430 2024-05-04T12:18:33.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.390+0000 7f07ad7fa640 1 --2- 172.21.15.52:0/90586359 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f079c0678c0 0x7f079c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.390+0000 7f07ad7fa640 1 -- 172.21.15.52:0/90586359 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f07a0085110 con 0x7f07b006e430 2024-05-04T12:18:33.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.390+0000 7f07affff640 1 --2- 172.21.15.52:0/90586359 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f079c0678c0 0x7f079c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.390+0000 7f07affff640 1 --2- 172.21.15.52:0/90586359 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f079c0678c0 0x7f079c069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f07a40ae000 tx=0x7f07a40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:33.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.394+0000 7f07ad7fa640 1 -- 172.21.15.52:0/90586359 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f07a0055c50 con 0x7f07b006e430 2024-05-04T12:18:33.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.582+0000 7f07b701e640 1 -- 172.21.15.52:0/90586359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07b0075320 con 0x7f07b006e430 2024-05-04T12:18:33.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.582+0000 7f07ad7fa640 1 -- 172.21.15.52:0/90586359 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7f07a0055550 con 0x7f07b006e430 2024-05-04T12:18:33.585 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:33.585 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6836/482177111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":482177111},{"type":"v1","addr":"172.21.15.52:6836","nonce":482177111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/163453709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6835","nonce":163453709},{"type":"v1","addr":"172.21.15.52:6837","nonce":163453709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3596668218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3596668218},{"type":"v1","addr":"172.21.15.111:6833","nonce":3596668218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-04T12:18:22.189525+0000","modified":"2024-05-04T12:18:25.240402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/1656435333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1656435333},{"type":"v1","addr":"172.21.15.111:6835","nonce":1656435333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-05-04T12:18:33.585 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 7 2024-05-04T12:18:33.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.586+0000 7f07b701e640 1 -- 172.21.15.52:0/90586359 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f079c0678c0 msgr2=0x7f079c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.586+0000 7f07b701e640 1 --2- 172.21.15.52:0/90586359 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f079c0678c0 0x7f079c069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f07a40ae000 tx=0x7f07a40a7000 comp rx=0 tx=0).stop 2024-05-04T12:18:33.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.586+0000 7f07b701e640 1 -- 172.21.15.52:0/90586359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07b006e430 msgr2=0x7f07b014f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.586+0000 7f07b701e640 1 --2- 172.21.15.52:0/90586359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07b006e430 0x7f07b014f850 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f07a000c370 tx=0x7f07a000c830 comp rx=0 tx=0).stop 2024-05-04T12:18:33.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.586+0000 7f07b701e640 1 -- 172.21.15.52:0/90586359 shutdown_connections 2024-05-04T12:18:33.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.586+0000 7f07b701e640 1 --2- 172.21.15.52:0/90586359 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f079c0678c0 0x7f079c069d80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.586+0000 7f07b701e640 1 --2- 172.21.15.52:0/90586359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07b013dec0 0x7f07b015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.586+0000 7f07b701e640 1 --2- 172.21.15.52:0/90586359 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f07b0073590 0x7f07b015a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.586+0000 7f07b701e640 1 --2- 172.21.15.52:0/90586359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07b006e430 0x7f07b014f850 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.586+0000 7f07b701e640 1 -- 172.21.15.52:0/90586359 >> 172.21.15.52:0/90586359 conn(0x7f07b0067070 msgr2=0x7f07b013d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:33.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.586+0000 7f07b701e640 1 -- 172.21.15.52:0/90586359 shutdown_connections 2024-05-04T12:18:33.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.586+0000 7f07b701e640 1 -- 172.21.15.52:0/90586359 wait complete. 2024-05-04T12:18:33.603 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:33.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6fd61640 1 Processor -- start 2024-05-04T12:18:33.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6fd61640 1 -- start start 2024-05-04T12:18:33.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6fd61640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5c6813ee60 0x7f5c6815d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6fd61640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5c6813f240 0x7f5c6806b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6fd61640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5c6806b750 0x7f5c6815fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6fd61640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f5c68160770 con 0x7f5c6806b750 2024-05-04T12:18:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6fd61640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f5c68160a50 con 0x7f5c6813f240 2024-05-04T12:18:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6fd61640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6815c680 con 0x7f5c6813ee60 2024-05-04T12:18:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6e2fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5c6806b750 0x7f5c6815fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6e2fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5c6806b750 0x7f5c6815fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53858/0 (socket says 172.21.15.52:53858) 2024-05-04T12:18:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6e2fe640 1 -- 172.21.15.52:0/1186441743 learned_addr learned my addr 172.21.15.52:0/1186441743 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6dafd640 1 --2- 172.21.15.52:0/1186441743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5c6813ee60 0x7f5c6815d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6d2fc640 1 --2- 172.21.15.52:0/1186441743 >> v2:172.21.15.111:3300/0 conn(0x7f5c6813f240 0x7f5c6806b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6e2fe640 1 -- 172.21.15.52:0/1186441743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5c6813ee60 msgr2=0x7f5c6815d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6e2fe640 1 --2- 172.21.15.52:0/1186441743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5c6813ee60 0x7f5c6815d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6e2fe640 1 -- 172.21.15.52:0/1186441743 >> v2:172.21.15.111:3300/0 conn(0x7f5c6813f240 msgr2=0x7f5c6806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:33.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6e2fe640 1 --2- 172.21.15.52:0/1186441743 >> v2:172.21.15.111:3300/0 conn(0x7f5c6813f240 0x7f5c6806b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.714+0000 7f5c6e2fe640 1 -- 172.21.15.52:0/1186441743 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6815c960 con 0x7f5c6806b750 2024-05-04T12:18:33.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6e2fe640 1 --2- 172.21.15.52:0/1186441743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c6806b750 0x7f5c6815fb60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c6806e9d0 tx=0x7f5c6001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d72a32b6ca4fb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:33.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6cafb640 1 -- 172.21.15.52:0/1186441743 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c600a5670 con 0x7f5c6806b750 2024-05-04T12:18:33.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6cafb640 1 -- 172.21.15.52:0/1186441743 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c600a5810 con 0x7f5c6806b750 2024-05-04T12:18:33.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6cafb640 1 -- 172.21.15.52:0/1186441743 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c600a5be0 con 0x7f5c6806b750 2024-05-04T12:18:33.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1186441743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c6806b750 msgr2=0x7f5c6815fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 --2- 172.21.15.52:0/1186441743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c6806b750 0x7f5c6815fb60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c6806e9d0 tx=0x7f5c6001c9f0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1186441743 shutdown_connections 2024-05-04T12:18:33.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 --2- 172.21.15.52:0/1186441743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c6806b750 0x7f5c6815fb60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 --2- 172.21.15.52:0/1186441743 >> v2:172.21.15.111:3300/0 conn(0x7f5c6813f240 0x7f5c6806b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 --2- 172.21.15.52:0/1186441743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5c6813ee60 0x7f5c6815d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1186441743 >> 172.21.15.52:0/1186441743 conn(0x7f5c68070690 msgr2=0x7f5c68070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:33.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1186441743 shutdown_connections 2024-05-04T12:18:33.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1186441743 wait complete. 2024-05-04T12:18:33.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 Processor -- start 2024-05-04T12:18:33.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 -- start start 2024-05-04T12:18:33.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c6806b750 0x7f5c68152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c6813ee60 0x7f5c6813c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5c6813f240 0x7f5c681545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5c680d0f70 con 0x7f5c6813ee60 2024-05-04T12:18:33.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5c680d0df0 con 0x7f5c6806b750 2024-05-04T12:18:33.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5c680d10f0 con 0x7f5c6813f240 2024-05-04T12:18:33.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6d2fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c6813ee60 0x7f5c6813c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6d2fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c6813ee60 0x7f5c6813c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53860/0 (socket says 172.21.15.52:53860) 2024-05-04T12:18:33.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6d2fc640 1 -- 172.21.15.52:0/1422241106 learned_addr learned my addr 172.21.15.52:0/1422241106 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:33.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6dafd640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c6806b750 0x7f5c68152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6e2fe640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5c6813f240 0x7f5c681545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6d2fc640 1 -- 172.21.15.52:0/1422241106 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5c6813f240 msgr2=0x7f5c681545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6d2fc640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5c6813f240 0x7f5c681545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6d2fc640 1 -- 172.21.15.52:0/1422241106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c6806b750 msgr2=0x7f5c68152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6d2fc640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c6806b750 0x7f5c68152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6d2fc640 1 -- 172.21.15.52:0/1422241106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c681fa7e0 con 0x7f5c6813ee60 2024-05-04T12:18:33.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6dafd640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c6806b750 0x7f5c68152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:33.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6d2fc640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c6813ee60 0x7f5c6813c960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800d620 tx=0x7f5c5800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:33.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c5e7fc640 1 -- 172.21.15.52:0/1422241106 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5800b620 con 0x7f5c6813ee60 2024-05-04T12:18:33.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c5e7fc640 1 -- 172.21.15.52:0/1422241106 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c58019070 con 0x7f5c6813ee60 2024-05-04T12:18:33.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1422241106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c681faac0 con 0x7f5c6813ee60 2024-05-04T12:18:33.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c5e7fc640 1 -- 172.21.15.52:0/1422241106 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5800b9e0 con 0x7f5c6813ee60 2024-05-04T12:18:33.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.718+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1422241106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c681fb080 con 0x7f5c6813ee60 2024-05-04T12:18:33.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.722+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1422241106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c68062e50 con 0x7f5c6813ee60 2024-05-04T12:18:33.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.722+0000 7f5c5e7fc640 1 -- 172.21.15.52:0/1422241106 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5c5800bb80 con 0x7f5c6813ee60 2024-05-04T12:18:33.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.722+0000 7f5c5e7fc640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5c4c0678c0 0x7f5c4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:33.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.722+0000 7f5c5e7fc640 1 -- 172.21.15.52:0/1422241106 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5c58086000 con 0x7f5c6813ee60 2024-05-04T12:18:33.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.722+0000 7f5c6dafd640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5c4c0678c0 0x7f5c4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:33.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.722+0000 7f5c6dafd640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5c4c0678c0 0x7f5c4c069d80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5c681fa740 tx=0x7f5c64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:33.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.726+0000 7f5c5e7fc640 1 -- 172.21.15.52:0/1422241106 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5c58056bd0 con 0x7f5c6813ee60 2024-05-04T12:18:33.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.886+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1422241106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c68075320 con 0x7f5c6813ee60 2024-05-04T12:18:33.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.886+0000 7f5c5e7fc640 1 -- 172.21.15.52:0/1422241106 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f5c580564d0 con 0x7f5c6813ee60 2024-05-04T12:18:33.890 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:33.891 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.890+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1422241106 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5c4c0678c0 msgr2=0x7f5c4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.890+0000 7f5c6fd61640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5c4c0678c0 0x7f5c4c069d80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5c681fa740 tx=0x7f5c64008040 comp rx=0 tx=0).stop 2024-05-04T12:18:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.890+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1422241106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c6813ee60 msgr2=0x7f5c6813c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.890+0000 7f5c6fd61640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c6813ee60 0x7f5c6813c960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800d620 tx=0x7f5c5800dae0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.890+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1422241106 shutdown_connections 2024-05-04T12:18:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.890+0000 7f5c6fd61640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5c4c0678c0 0x7f5c4c069d80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.890+0000 7f5c6fd61640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5c6813f240 0x7f5c681545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.890+0000 7f5c6fd61640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c6813ee60 0x7f5c6813c960 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.890+0000 7f5c6fd61640 1 --2- 172.21.15.52:0/1422241106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c6806b750 0x7f5c68152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:33.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.890+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1422241106 >> 172.21.15.52:0/1422241106 conn(0x7f5c68070690 msgr2=0x7f5c680a9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:33.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.890+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1422241106 shutdown_connections 2024-05-04T12:18:33.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:33.890+0000 7f5c6fd61640 1 -- 172.21.15.52:0/1422241106 wait complete. 2024-05-04T12:18:33.910 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T12:18:34.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc01a6a1640 1 Processor -- start 2024-05-04T12:18:34.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc01a6a1640 1 -- start start 2024-05-04T12:18:34.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc01a6a1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc01415c7f0 0x7fc0140723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc01a6a1640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0140cd190 0x7fc01415c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc01a6a1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc014072aa0 con 0x7fc0140ccdb0 2024-05-04T12:18:34.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc01a6a1640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc014070690 con 0x7fc01415c7f0 2024-05-04T12:18:34.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc01a6a1640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc0140708e0 con 0x7fc0140cd190 2024-05-04T12:18:34.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc013fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc0140ccdb0 0x7fc014158640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51672/0 (socket says 172.21.15.52:51672) 2024-05-04T12:18:34.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc013fff640 1 -- 172.21.15.52:0/3548700893 learned_addr learned my addr 172.21.15.52:0/3548700893 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:34.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc00b7fe640 1 --2- 172.21.15.52:0/3548700893 >> v2:172.21.15.52:3300/0 conn(0x7fc01415c7f0 0x7fc0140723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc018c3e640 1 --2- 172.21.15.52:0/3548700893 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0140cd190 0x7fc01415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc0137fe640 1 -- 172.21.15.52:0/3548700893 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2964202583 0 0) 0x7fc014072aa0 con 0x7fc0140ccdb0 2024-05-04T12:18:34.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc0137fe640 1 -- 172.21.15.52:0/3548700893 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc000003530 con 0x7fc0140ccdb0 2024-05-04T12:18:34.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc00b7fe640 1 -- 172.21.15.52:0/3548700893 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0140cd190 msgr2=0x7fc01415c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:34.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc00b7fe640 1 --2- 172.21.15.52:0/3548700893 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0140cd190 0x7fc01415c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc00b7fe640 1 -- 172.21.15.52:0/3548700893 >> v1:172.21.15.52:6789/0 conn(0x7fc0140ccdb0 legacy=0x7fc014158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.030+0000 7fc00b7fe640 1 -- 172.21.15.52:0/3548700893 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc014070bc0 con 0x7fc01415c7f0 2024-05-04T12:18:34.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc00b7fe640 1 --2- 172.21.15.52:0/3548700893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc01415c7f0 0x7fc0140723e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0097c0 tx=0x7fbffc01e620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7877263fcb059250 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:34.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc0137fe640 1 -- 172.21.15.52:0/3548700893 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0b2070 con 0x7fc01415c7f0 2024-05-04T12:18:34.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc0137fe640 1 -- 172.21.15.52:0/3548700893 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc0ad600 con 0x7fc01415c7f0 2024-05-04T12:18:34.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc0137fe640 1 -- 172.21.15.52:0/3548700893 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0ad9d0 con 0x7fc01415c7f0 2024-05-04T12:18:34.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 -- 172.21.15.52:0/3548700893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc01415c7f0 msgr2=0x7fc0140723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 --2- 172.21.15.52:0/3548700893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc01415c7f0 0x7fc0140723e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0097c0 tx=0x7fbffc01e620 comp rx=0 tx=0).stop 2024-05-04T12:18:34.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 -- 172.21.15.52:0/3548700893 shutdown_connections 2024-05-04T12:18:34.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 --2- 172.21.15.52:0/3548700893 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0140cd190 0x7fc01415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 --2- 172.21.15.52:0/3548700893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc01415c7f0 0x7fc0140723e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 -- 172.21.15.52:0/3548700893 >> 172.21.15.52:0/3548700893 conn(0x7fc0140b14a0 msgr2=0x7fc0140b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:34.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 -- 172.21.15.52:0/3548700893 shutdown_connections 2024-05-04T12:18:34.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 -- 172.21.15.52:0/3548700893 wait complete. 2024-05-04T12:18:34.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 Processor -- start 2024-05-04T12:18:34.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 -- start start 2024-05-04T12:18:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0140ccdb0 0x7fc01414eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc0140cd190 0x7fc01415dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc01415c7f0 0x7fc0140ab900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc014166e20 con 0x7fc0140ccdb0 2024-05-04T12:18:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc014166ca0 con 0x7fc0140cd190 2024-05-04T12:18:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc014166fa0 con 0x7fc01415c7f0 2024-05-04T12:18:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc013fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0140ccdb0 0x7fc01414eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc013fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0140ccdb0 0x7fc01414eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53882/0 (socket says 172.21.15.52:53882) 2024-05-04T12:18:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc013fff640 1 -- 172.21.15.52:0/1658933432 learned_addr learned my addr 172.21.15.52:0/1658933432 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc018c3e640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc01415c7f0 0x7fc0140ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc00b7fe640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc0140cd190 0x7fc01415dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc013fff640 1 -- 172.21.15.52:0/1658933432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc01415c7f0 msgr2=0x7fc0140ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc013fff640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc01415c7f0 0x7fc0140ab900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc013fff640 1 -- 172.21.15.52:0/1658933432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc0140cd190 msgr2=0x7fc01415dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc013fff640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc0140cd190 0x7fc01415dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc013fff640 1 -- 172.21.15.52:0/1658933432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0141412d0 con 0x7fc0140ccdb0 2024-05-04T12:18:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc018c3e640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc01415c7f0 0x7fc0140ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc00b7fe640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc0140cd190 0x7fc01415dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc013fff640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0140ccdb0 0x7fc01414eff0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc00400c3f0 tx=0x7fc00400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc0117fa640 1 -- 172.21.15.52:0/1658933432 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004017070 con 0x7fc0140ccdb0 2024-05-04T12:18:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc0117fa640 1 -- 172.21.15.52:0/1658933432 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc004005cf0 con 0x7fc0140ccdb0 2024-05-04T12:18:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.034+0000 7fc01a6a1640 1 -- 172.21.15.52:0/1658933432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01414f6f0 con 0x7fc0140ccdb0 2024-05-04T12:18:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.038+0000 7fc0117fa640 1 -- 172.21.15.52:0/1658933432 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0040072f0 con 0x7fc0140ccdb0 2024-05-04T12:18:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.038+0000 7fc01a6a1640 1 -- 172.21.15.52:0/1658933432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01414fcb0 con 0x7fc0140ccdb0 2024-05-04T12:18:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.038+0000 7fc01a6a1640 1 -- 172.21.15.52:0/1658933432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc014062e50 con 0x7fc0140ccdb0 2024-05-04T12:18:34.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.038+0000 7fc0117fa640 1 -- 172.21.15.52:0/1658933432 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc00401a070 con 0x7fc0140ccdb0 2024-05-04T12:18:34.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.038+0000 7fc0117fa640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbff40678c0 0x7fbff4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.038+0000 7fc0117fa640 1 -- 172.21.15.52:0/1658933432 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc004085120 con 0x7fc0140ccdb0 2024-05-04T12:18:34.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.038+0000 7fc00b7fe640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbff40678c0 0x7fbff4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.038+0000 7fc00b7fe640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbff40678c0 0x7fbff4069d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0af000 tx=0x7fbffc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:34.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.042+0000 7fc0117fa640 1 -- 172.21.15.52:0/1658933432 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc0141412d0 con 0x7fc0140ccdb0 2024-05-04T12:18:34.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.242+0000 7fc01a6a1640 1 -- 172.21.15.52:0/1658933432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0140b0fa0 con 0x7fc0140ccdb0 2024-05-04T12:18:34.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.246+0000 7fc0117fa640 1 -- 172.21.15.52:0/1658933432 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7fc004055c60 con 0x7fc0140ccdb0 2024-05-04T12:18:34.247 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:34.247 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6836/482177111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":482177111},{"type":"v1","addr":"172.21.15.52:6836","nonce":482177111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/163453709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6835","nonce":163453709},{"type":"v1","addr":"172.21.15.52:6837","nonce":163453709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3596668218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3596668218},{"type":"v1","addr":"172.21.15.111:6833","nonce":3596668218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-04T12:18:22.189525+0000","modified":"2024-05-04T12:18:25.240402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/1656435333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1656435333},{"type":"v1","addr":"172.21.15.111:6835","nonce":1656435333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-05-04T12:18:34.247 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 7 2024-05-04T12:18:34.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.246+0000 7fc01a6a1640 1 -- 172.21.15.52:0/1658933432 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbff40678c0 msgr2=0x7fbff4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.246+0000 7fc01a6a1640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbff40678c0 0x7fbff4069d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0af000 tx=0x7fbffc008040 comp rx=0 tx=0).stop 2024-05-04T12:18:34.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.246+0000 7fc01a6a1640 1 -- 172.21.15.52:0/1658933432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0140ccdb0 msgr2=0x7fc01414eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.246+0000 7fc01a6a1640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0140ccdb0 0x7fc01414eff0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc00400c3f0 tx=0x7fc00400c8b0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.246+0000 7fc01a6a1640 1 -- 172.21.15.52:0/1658933432 shutdown_connections 2024-05-04T12:18:34.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.246+0000 7fc01a6a1640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbff40678c0 0x7fbff4069d80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.246+0000 7fc01a6a1640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc01415c7f0 0x7fc0140ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.246+0000 7fc01a6a1640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc0140cd190 0x7fc01415dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.246+0000 7fc01a6a1640 1 --2- 172.21.15.52:0/1658933432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0140ccdb0 0x7fc01414eff0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.246+0000 7fc01a6a1640 1 -- 172.21.15.52:0/1658933432 >> 172.21.15.52:0/1658933432 conn(0x7fc0140b14a0 msgr2=0x7fc014157ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:34.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.246+0000 7fc01a6a1640 1 -- 172.21.15.52:0/1658933432 shutdown_connections 2024-05-04T12:18:34.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.246+0000 7fc01a6a1640 1 -- 172.21.15.52:0/1658933432 wait complete. 2024-05-04T12:18:34.669 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-04T12:18:22.189525+0000', 'modified': '2024-05-04T12:18:25.240402+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6835/1656435333', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 1656435333}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 1656435333}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4282, 'qdb_cluster': [4282]} 2024-05-04T12:18:34.669 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-04T12:18:34.670 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-04T12:18:34.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.802+0000 7f854723e640 1 Processor -- start 2024-05-04T12:18:34.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.802+0000 7f854723e640 1 -- start start 2024-05-04T12:18:34.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.802+0000 7f854723e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f854013f000 0x7f854006b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.802+0000 7f854723e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f854006b750 0x7f854015fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f854723e640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f854013f3e0 con 0x7f8540160090 2024-05-04T12:18:34.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f854723e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f854006bbb0 con 0x7f854013f000 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f854723e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f854015c560 con 0x7f854006b750 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f85457db640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f8540160090 0x7f85400ceda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51682/0 (socket says 172.21.15.52:51682) 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f85457db640 1 -- 172.21.15.52:0/410903425 learned_addr learned my addr 172.21.15.52:0/410903425 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f8544fda640 1 --2- 172.21.15.52:0/410903425 >> v2:172.21.15.111:3300/0 conn(0x7f854013f000 0x7f854006b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853ffff640 1 --2- 172.21.15.52:0/410903425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f854006b750 0x7f854015fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 -- 172.21.15.52:0/410903425 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 205091406 0 0) 0x7f854013f3e0 con 0x7f8540160090 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 -- 172.21.15.52:0/410903425 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8530003530 con 0x7f8540160090 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 -- 172.21.15.52:0/410903425 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2527651689 0 0) 0x7f8530003530 con 0x7f8540160090 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 -- 172.21.15.52:0/410903425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f854006b750 msgr2=0x7f854015fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 --2- 172.21.15.52:0/410903425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f854006b750 0x7f854015fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 -- 172.21.15.52:0/410903425 >> v2:172.21.15.111:3300/0 conn(0x7f854013f000 msgr2=0x7f854006b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 --2- 172.21.15.52:0/410903425 >> v2:172.21.15.111:3300/0 conn(0x7f854013f000 0x7f854006b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 -- 172.21.15.52:0/410903425 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85401608c0 con 0x7f8540160090 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 -- 172.21.15.52:0/410903425 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f8538001a00 con 0x7f8540160090 2024-05-04T12:18:34.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 -- 172.21.15.52:0/410903425 >> v1:172.21.15.52:6789/0 conn(0x7f8540160090 legacy=0x7f85400ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 --2- 172.21.15.52:0/410903425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85300039c0 0x7f8530003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 --2- 172.21.15.52:0/410903425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8530004810 0x7f8530004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 --2- 172.21.15.52:0/410903425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85300051d0 0x7f853000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 -- 172.21.15.52:0/410903425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f854015c560 con 0x7f85300039c0 2024-05-04T12:18:34.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 -- 172.21.15.52:0/410903425 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f853000bf70 con 0x7f85300051d0 2024-05-04T12:18:34.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 -- 172.21.15.52:0/410903425 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f853000c170 con 0x7f8530004810 2024-05-04T12:18:34.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853ffff640 1 --2- 172.21.15.52:0/410903425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f854006b750 0x7f854015fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:34.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f8544fda640 1 --2- 172.21.15.52:0/410903425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85300039c0 0x7f8530003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f85457db640 1 --2- 172.21.15.52:0/410903425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85300051d0 0x7f853000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f8544fda640 1 -- 172.21.15.52:0/410903425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8530004810 msgr2=0x7f8530004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:34.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f8544fda640 1 --2- 172.21.15.52:0/410903425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8530004810 0x7f8530004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f8544fda640 1 -- 172.21.15.52:0/410903425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85300051d0 msgr2=0x7f853000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:34.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f8544fda640 1 --2- 172.21.15.52:0/410903425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85300051d0 0x7f853000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f8544fda640 1 -- 172.21.15.52:0/410903425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8530003530 con 0x7f85300039c0 2024-05-04T12:18:34.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f8544fda640 1 --2- 172.21.15.52:0/410903425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85300039c0 0x7f8530003de0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8534009ae0 tx=0x7f853401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dc9a262aa9ce49 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:34.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 -- 172.21.15.52:0/410903425 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85340096b0 con 0x7f85300039c0 2024-05-04T12:18:34.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 -- 172.21.15.52:0/410903425 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85340a6490 con 0x7f85300039c0 2024-05-04T12:18:34.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.806+0000 7f853f7fe640 1 -- 172.21.15.52:0/410903425 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85340a6880 con 0x7f85300039c0 2024-05-04T12:18:34.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 -- 172.21.15.52:0/410903425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85300039c0 msgr2=0x7f8530003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 --2- 172.21.15.52:0/410903425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85300039c0 0x7f8530003de0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8534009ae0 tx=0x7f853401cab0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f85457db640 1 -- 172.21.15.52:0/410903425 reap_dead start 2024-05-04T12:18:34.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 -- 172.21.15.52:0/410903425 shutdown_connections 2024-05-04T12:18:34.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 -- 172.21.15.52:0/410903425 >> 172.21.15.52:0/410903425 conn(0x7f8540070690 msgr2=0x7f8540070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:34.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 -- 172.21.15.52:0/410903425 shutdown_connections 2024-05-04T12:18:34.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 -- 172.21.15.52:0/410903425 wait complete. 2024-05-04T12:18:34.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 Processor -- start 2024-05-04T12:18:34.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 -- start start 2024-05-04T12:18:34.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85300039c0 0x7f854013d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8540144240 0x7f8540145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f854015e7b0 0x7f854015ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8540160620 con 0x7f85300039c0 2024-05-04T12:18:34.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f85401604a0 con 0x7f854015e7b0 2024-05-04T12:18:34.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f854015d300 con 0x7f8540144240 2024-05-04T12:18:34.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f8544fda640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85300039c0 0x7f854013d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f8544fda640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85300039c0 0x7f854013d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53892/0 (socket says 172.21.15.52:53892) 2024-05-04T12:18:34.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f8544fda640 1 -- 172.21.15.52:0/2753319823 learned_addr learned my addr 172.21.15.52:0/2753319823 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:34.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f85457db640 1 --2- 172.21.15.52:0/2753319823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f854015e7b0 0x7f854015ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f853ffff640 1 --2- 172.21.15.52:0/2753319823 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8540144240 0x7f8540145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f8544fda640 1 -- 172.21.15.52:0/2753319823 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8540144240 msgr2=0x7f8540145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f8544fda640 1 --2- 172.21.15.52:0/2753319823 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8540144240 0x7f8540145030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f8544fda640 1 -- 172.21.15.52:0/2753319823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f854015e7b0 msgr2=0x7f854015ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f8544fda640 1 --2- 172.21.15.52:0/2753319823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f854015e7b0 0x7f854015ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f8544fda640 1 -- 172.21.15.52:0/2753319823 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85400b0b40 con 0x7f85300039c0 2024-05-04T12:18:34.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f8544fda640 1 --2- 172.21.15.52:0/2753319823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85300039c0 0x7f854013d520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f853401c8e0 tx=0x7f8530005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:34.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f853d7fa640 1 -- 172.21.15.52:0/2753319823 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85340bf070 con 0x7f85300039c0 2024-05-04T12:18:34.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f853d7fa640 1 -- 172.21.15.52:0/2753319823 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f853400f000 con 0x7f85300039c0 2024-05-04T12:18:34.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 -- 172.21.15.52:0/2753319823 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8540141e70 con 0x7f85300039c0 2024-05-04T12:18:34.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f853d7fa640 1 -- 172.21.15.52:0/2753319823 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85340a6e60 con 0x7f85300039c0 2024-05-04T12:18:34.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.810+0000 7f854723e640 1 -- 172.21.15.52:0/2753319823 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8540142430 con 0x7f85300039c0 2024-05-04T12:18:34.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f853d7fa640 1 -- 172.21.15.52:0/2753319823 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f85340c2070 con 0x7f85300039c0 2024-05-04T12:18:34.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f853d7fa640 1 --2- 172.21.15.52:0/2753319823 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f85240678c0 0x7f8524069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f853d7fa640 1 -- 172.21.15.52:0/2753319823 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f853411f4b0 con 0x7f85300039c0 2024-05-04T12:18:34.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f853ffff640 1 --2- 172.21.15.52:0/2753319823 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f85240678c0 0x7f8524069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f854a3b7000 1 Processor -- start 2024-05-04T12:18:34.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f854a3b7000 1 -- start start 2024-05-04T12:18:34.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f853ffff640 1 --2- 172.21.15.52:0/2753319823 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f85240678c0 0x7f8524069d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f852c006aa0 tx=0x7f852c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f854a3b7000 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aecb3ef0 0x5595aecb42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f854a3b7000 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aecb4810 0x5595aec6a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f854a3b7000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aec6abc0 0x5595aec6afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f854a3b7000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5595aeab85e0 con 0x5595aec6abc0 2024-05-04T12:18:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f854a3b7000 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5595aea3fc50 con 0x5595aecb3ef0 2024-05-04T12:18:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f854a3b7000 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x5595aeab45a0 con 0x5595aecb4810 2024-05-04T12:18:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f85457db640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aec6abc0 0x5595aec6afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f85457db640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aec6abc0 0x5595aec6afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53904/0 (socket says 172.21.15.52:53904) 2024-05-04T12:18:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f85457db640 1 -- 172.21.15.52:0/1384209584 learned_addr learned my addr 172.21.15.52:0/1384209584 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f8544fda640 1 --2- 172.21.15.52:0/1384209584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aecb3ef0 0x5595aecb42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.814+0000 7f853ffff640 1 --2- 172.21.15.52:0/1384209584 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aecb4810 0x5595aec6a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 -- 172.21.15.52:0/1384209584 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aecb4810 msgr2=0x5595aec6a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 --2- 172.21.15.52:0/1384209584 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aecb4810 0x5595aec6a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 -- 172.21.15.52:0/1384209584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aecb3ef0 msgr2=0x5595aecb42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 --2- 172.21.15.52:0/1384209584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aecb3ef0 0x5595aecb42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 -- 172.21.15.52:0/1384209584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595aea3dd10 con 0x5595aec6abc0 2024-05-04T12:18:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f853ffff640 1 --2- 172.21.15.52:0/1384209584 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aecb4810 0x5595aec6a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 --2- 172.21.15.52:0/1384209584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aec6abc0 0x5595aec6afa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8540160470 tx=0x7f8538001dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6158eb61a4289ac8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f852a7fc640 1 -- 172.21.15.52:0/1384209584 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538014400 con 0x5595aec6abc0 2024-05-04T12:18:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f852a7fc640 1 -- 172.21.15.52:0/1384209584 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85380145a0 con 0x5595aec6abc0 2024-05-04T12:18:34.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f852a7fc640 1 -- 172.21.15.52:0/1384209584 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538014950 con 0x5595aec6abc0 2024-05-04T12:18:34.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 -- 172.21.15.52:0/1384209584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aec6abc0 msgr2=0x5595aec6afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 --2- 172.21.15.52:0/1384209584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aec6abc0 0x5595aec6afa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8540160470 tx=0x7f8538001dd0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 -- 172.21.15.52:0/1384209584 shutdown_connections 2024-05-04T12:18:34.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 --2- 172.21.15.52:0/1384209584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aec6abc0 0x5595aec6afa0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 --2- 172.21.15.52:0/1384209584 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aecb4810 0x5595aec6a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 --2- 172.21.15.52:0/1384209584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aecb3ef0 0x5595aecb42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 -- 172.21.15.52:0/1384209584 >> 172.21.15.52:0/1384209584 conn(0x5595aec66b70 msgr2=0x5595aec68f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:34.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 -- 172.21.15.52:0/1384209584 shutdown_connections 2024-05-04T12:18:34.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 -- 172.21.15.52:0/1384209584 wait complete. 2024-05-04T12:18:34.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 -- start start 2024-05-04T12:18:34.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 Processor -- start 2024-05-04T12:18:34.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aec6abc0 0x5595aec98f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aecb3ef0 0x5595aec99460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb4810 0x5595aec999a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5595aea72520 con 0x5595aecb4810 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5595aea5e810 con 0x5595aec6abc0 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x5595aec669f0 con 0x5595aecb3ef0 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 10 client.0 _ll_register_callbacks cb 0x5595aec555c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb4810 0x5595aec999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb4810 0x5595aec999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53908/0 (socket says 172.21.15.52:53908) 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 -- 172.21.15.52:0/2150905705 learned_addr learned my addr 172.21.15.52:0/2150905705 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 -- 172.21.15.52:0/2150905705 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aecb3ef0 msgr2=0x5595aec99460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 --2- 172.21.15.52:0/2150905705 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aecb3ef0 0x5595aec99460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 -- 172.21.15.52:0/2150905705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aec6abc0 msgr2=0x5595aec98f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f8544fda640 1 --2- 172.21.15.52:0/2150905705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aec6abc0 0x5595aec98f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 --2- 172.21.15.52:0/2150905705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aec6abc0 0x5595aec98f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 -- 172.21.15.52:0/2150905705 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595aea507b0 con 0x5595aecb4810 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f85457db640 1 --2- 172.21.15.52:0/2150905705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb4810 0x5595aec999a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8538009010 tx=0x7f852c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f93b82a17f2125d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f851f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-04T12:18:34.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f851f7fe640 1 -- 172.21.15.52:0/2150905705 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538022070 con 0x5595aecb4810 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f851f7fe640 1 -- 172.21.15.52:0/2150905705 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f853801e850 con 0x5595aecb4810 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 -- 172.21.15.52:0/2150905705 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85380145a0 con 0x5595aecb4810 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f851f7fe640 1 -- 172.21.15.52:0/2150905705 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f853802d050 con 0x5595aecb4810 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 10 client.4599 fetch_fsmap learned FSMap version 7 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f854a3b7000 1 -- 172.21.15.52:0/2150905705 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5595aea3fea0 con 0x5595aecb4810 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.818+0000 7f851f7fe640 1 -- 172.21.15.52:0/2150905705 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85380145a0 con 0x5595aecb4810 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f851f7fe640 1 -- 172.21.15.52:0/2150905705 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7f8538030020 con 0x5595aecb4810 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 10 client.4599 fetch_fsmap finished waiting for FSMap version 7 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 10 client.4599 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 20 client.4599 populate_metadata read hostname 'smithi052' 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 1 --2- 172.21.15.52:0/2150905705 >> [v2:172.21.15.111:6834/1656435333,v1:172.21.15.111:6835/1656435333] conn(0x5595aecc3f90 0x5595aecc6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 4 client.4599 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 1 -- 172.21.15.52:0/2150905705 --> [v2:172.21.15.111:6834/1656435333,v1:172.21.15.111:6835/1656435333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595aea53ab0 con 0x5595aecc3f90 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f8544fda640 1 --2- 172.21.15.52:0/2150905705 >> [v2:172.21.15.111:6834/1656435333,v1:172.21.15.111:6835/1656435333] conn(0x5595aecc3f90 0x5595aecc6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f8544fda640 1 --2- 172.21.15.52:0/2150905705 >> [v2:172.21.15.111:6834/1656435333,v1:172.21.15.111:6835/1656435333] conn(0x5595aecc3f90 0x5595aecc6380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20705c07df93153d server_cookie=e20ad3de618f922b in_seq=0 out_seq=0 2024-05-04T12:18:34.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f851f7fe640 10 client.4599 ms_handle_connect on v2:172.21.15.111:6834/1656435333 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f851f7fe640 1 -- 172.21.15.52:0/2150905705 <== mds.0 v2:172.21.15.111:6834/1656435333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595aea53ab0 con 0x5595aecc3f90 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f851f7fe640 10 client.4599 handle_command_reply: tid=0 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 1 client.4599 shutdown 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 1 -- 172.21.15.52:0/2150905705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb4810 msgr2=0x5595aec999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 1 --2- 172.21.15.52:0/2150905705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb4810 0x5595aec999a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8538009010 tx=0x7f852c004740 comp rx=0 tx=0).stop 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 1 -- 172.21.15.52:0/2150905705 shutdown_connections 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 1 --2- 172.21.15.52:0/2150905705 >> [v2:172.21.15.111:6834/1656435333,v1:172.21.15.111:6835/1656435333] conn(0x5595aecc3f90 0x5595aecc6380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 1 --2- 172.21.15.52:0/2150905705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb4810 0x5595aec999a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 1 --2- 172.21.15.52:0/2150905705 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aecb3ef0 0x5595aec99460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f851f7fe640 0 client.4599 ms_handle_reset on v2:172.21.15.111:6834/1656435333 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 1 --2- 172.21.15.52:0/2150905705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aec6abc0 0x5595aec98f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 1 -- 172.21.15.52:0/2150905705 >> 172.21.15.52:0/2150905705 conn(0x5595aec66b70 msgr2=0x5595aec697f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 1 -- 172.21.15.52:0/2150905705 shutdown_connections 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 1 -- 172.21.15.52:0/2150905705 wait complete. 2024-05-04T12:18:34.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.822+0000 7f854a3b7000 20 client.4599 trim_cache size 0 max 16384 2024-05-04T12:18:34.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 Processor -- start 2024-05-04T12:18:34.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 -- start start 2024-05-04T12:18:34.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aec6abc0 0x5595aec68f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb3ef0 0x5595aec694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aecb4810 0x5595aec699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8540202eb0 con 0x5595aecb3ef0 2024-05-04T12:18:34.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8540072670 con 0x5595aec6abc0 2024-05-04T12:18:34.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f85400b2c00 con 0x5595aecb4810 2024-05-04T12:18:34.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f853ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb3ef0 0x5595aec694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f853ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb3ef0 0x5595aec694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53916/0 (socket says 172.21.15.52:53916) 2024-05-04T12:18:34.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f853ffff640 1 -- 172.21.15.52:0/4232355539 learned_addr learned my addr 172.21.15.52:0/4232355539 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:34.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f853ffff640 1 -- 172.21.15.52:0/4232355539 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aecb4810 msgr2=0x5595aec699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:34.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f853ffff640 1 --2- 172.21.15.52:0/4232355539 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aecb4810 0x5595aec699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f853ffff640 1 -- 172.21.15.52:0/4232355539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aec6abc0 msgr2=0x5595aec68f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:34.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f853ffff640 1 --2- 172.21.15.52:0/4232355539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aec6abc0 0x5595aec68f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f853ffff640 1 -- 172.21.15.52:0/4232355539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595aea40e80 con 0x5595aecb3ef0 2024-05-04T12:18:34.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f853ffff640 1 --2- 172.21.15.52:0/4232355539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb3ef0 0x5595aec694a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f852c00c010 tx=0x7f852c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851a2f2adfb7bc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f8529ffb640 1 -- 172.21.15.52:0/4232355539 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c014ae0 con 0x5595aecb3ef0 2024-05-04T12:18:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f8529ffb640 1 -- 172.21.15.52:0/4232355539 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f852c026070 con 0x5595aecb3ef0 2024-05-04T12:18:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f8529ffb640 1 -- 172.21.15.52:0/4232355539 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c00fe90 con 0x5595aecb3ef0 2024-05-04T12:18:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 -- 172.21.15.52:0/4232355539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb3ef0 msgr2=0x5595aec694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 --2- 172.21.15.52:0/4232355539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb3ef0 0x5595aec694a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f852c00c010 tx=0x7f852c012000 comp rx=0 tx=0).stop 2024-05-04T12:18:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 -- 172.21.15.52:0/4232355539 shutdown_connections 2024-05-04T12:18:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 --2- 172.21.15.52:0/4232355539 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aecb4810 0x5595aec699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 --2- 172.21.15.52:0/4232355539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb3ef0 0x5595aec694a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 --2- 172.21.15.52:0/4232355539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aec6abc0 0x5595aec68f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 -- 172.21.15.52:0/4232355539 >> 172.21.15.52:0/4232355539 conn(0x5595aec66b70 msgr2=0x5595aecc14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 -- 172.21.15.52:0/4232355539 shutdown_connections 2024-05-04T12:18:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 -- 172.21.15.52:0/4232355539 wait complete. 2024-05-04T12:18:34.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 -- start start 2024-05-04T12:18:34.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.842+0000 7f854a3b7000 1 Processor -- start 2024-05-04T12:18:34.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aec6abc0 0x5595aed20660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aecb3ef0 0x5595aed20ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb4810 0x5595aed210e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5595aea6cc40 con 0x5595aecb4810 2024-05-04T12:18:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5595aeb6ae80 con 0x5595aecb3ef0 2024-05-04T12:18:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x5595aea81810 con 0x5595aec6abc0 2024-05-04T12:18:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 10 client.0 _ll_register_callbacks cb 0x7f8538001a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T12:18:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f85457db640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb4810 0x5595aed210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f85457db640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb4810 0x5595aed210e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53918/0 (socket says 172.21.15.52:53918) 2024-05-04T12:18:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f85457db640 1 -- 172.21.15.52:0/2960026988 learned_addr learned my addr 172.21.15.52:0/2960026988 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f853ffff640 1 --2- 172.21.15.52:0/2960026988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aecb3ef0 0x5595aed20ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f8544fda640 1 --2- 172.21.15.52:0/2960026988 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aec6abc0 0x5595aed20660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f85457db640 1 -- 172.21.15.52:0/2960026988 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aec6abc0 msgr2=0x5595aed20660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f85457db640 1 --2- 172.21.15.52:0/2960026988 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aec6abc0 0x5595aed20660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f85457db640 1 -- 172.21.15.52:0/2960026988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aecb3ef0 msgr2=0x5595aed20ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f85457db640 1 --2- 172.21.15.52:0/2960026988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aecb3ef0 0x5595aed20ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f85457db640 1 -- 172.21.15.52:0/2960026988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595aea85760 con 0x5595aecb4810 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f853ffff640 1 --2- 172.21.15.52:0/2960026988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aecb3ef0 0x5595aed20ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f85457db640 1 --2- 172.21.15.52:0/2960026988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb4810 0x5595aed210e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8538001eb0 tx=0x7f8538001e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b1ed955004a64c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f851f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f851f7fe640 1 -- 172.21.15.52:0/2960026988 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538022050 con 0x5595aecb4810 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f851f7fe640 1 -- 172.21.15.52:0/2960026988 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8538027640 con 0x5595aecb4810 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 -- 172.21.15.52:0/2960026988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f852c026070 con 0x5595aecb4810 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f851f7fe640 1 -- 172.21.15.52:0/2960026988 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f8538040050 con 0x5595aecb4810 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 10 client.4605 fetch_fsmap learned FSMap version 7 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 -- 172.21.15.52:0/2960026988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5595aeb119e0 con 0x5595aecb4810 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f851f7fe640 1 -- 172.21.15.52:0/2960026988 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f852c026070 con 0x5595aecb4810 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f851f7fe640 1 -- 172.21.15.52:0/2960026988 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7f8538043020 con 0x5595aecb4810 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 10 client.4605 fetch_fsmap finished waiting for FSMap version 7 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 10 client.4605 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 20 client.4605 populate_metadata read hostname 'smithi052' 2024-05-04T12:18:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 --2- 172.21.15.52:0/2960026988 >> [v2:172.21.15.111:6834/1656435333,v1:172.21.15.111:6835/1656435333] conn(0x5595aed21620 0x5595aed2bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 4 client.4605 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T12:18:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 -- 172.21.15.52:0/2960026988 --> [v2:172.21.15.111:6834/1656435333,v1:172.21.15.111:6835/1656435333] -- command(tid 0: {"prefix": "status"}) -- 0x5595aeac5b20 con 0x5595aed21620 2024-05-04T12:18:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f8544fda640 1 --2- 172.21.15.52:0/2960026988 >> [v2:172.21.15.111:6834/1656435333,v1:172.21.15.111:6835/1656435333] conn(0x5595aed21620 0x5595aed2bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f8544fda640 1 --2- 172.21.15.52:0/2960026988 >> [v2:172.21.15.111:6834/1656435333,v1:172.21.15.111:6835/1656435333] conn(0x5595aed21620 0x5595aed2bf80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc19a34d7e2801a server_cookie=bbf046c947b060b7 in_seq=0 out_seq=0 2024-05-04T12:18:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f851f7fe640 10 client.4605 ms_handle_connect on v2:172.21.15.111:6834/1656435333 2024-05-04T12:18:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f851f7fe640 1 -- 172.21.15.52:0/2960026988 <== mds.0 v2:172.21.15.111:6834/1656435333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5595aeac5b20 con 0x5595aed21620 2024-05-04T12:18:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f851f7fe640 10 client.4605 handle_command_reply: tid=0 2024-05-04T12:18:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 client.4605 shutdown 2024-05-04T12:18:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 -- 172.21.15.52:0/2960026988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb4810 msgr2=0x5595aed210e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 --2- 172.21.15.52:0/2960026988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb4810 0x5595aed210e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8538001eb0 tx=0x7f8538001e40 comp rx=0 tx=0).stop 2024-05-04T12:18:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 -- 172.21.15.52:0/2960026988 shutdown_connections 2024-05-04T12:18:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 --2- 172.21.15.52:0/2960026988 >> [v2:172.21.15.111:6834/1656435333,v1:172.21.15.111:6835/1656435333] conn(0x5595aed21620 0x5595aed2bf80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 --2- 172.21.15.52:0/2960026988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5595aecb4810 0x5595aed210e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 --2- 172.21.15.52:0/2960026988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595aecb3ef0 0x5595aed20ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 --2- 172.21.15.52:0/2960026988 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5595aec6abc0 0x5595aed20660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 -- 172.21.15.52:0/2960026988 >> 172.21.15.52:0/2960026988 conn(0x5595aec66b70 msgr2=0x5595aecb2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f851f7fe640 0 client.4605 ms_handle_reset on v2:172.21.15.111:6834/1656435333 2024-05-04T12:18:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 -- 172.21.15.52:0/2960026988 shutdown_connections 2024-05-04T12:18:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 1 -- 172.21.15.52:0/2960026988 wait complete. 2024-05-04T12:18:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.846+0000 7f854a3b7000 20 client.4605 trim_cache size 0 max 16384 2024-05-04T12:18:34.855 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-04T12:18:34.855 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "708d9177-9f3e-4c4b-9d29-7b98768b2a25", 2024-05-04T12:18:34.855 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-05-04T12:18:34.855 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 4282, 2024-05-04T12:18:34.856 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-05-04T12:18:34.856 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-05-04T12:18:34.856 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-05-04T12:18:34.856 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 12.024259132999999, 2024-05-04T12:18:34.856 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 7, 2024-05-04T12:18:34.856 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 21, 2024-05-04T12:18:34.856 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 21, 2024-05-04T12:18:34.856 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 18.763209423999999 2024-05-04T12:18:34.856 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-04T12:18:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.850+0000 7f854723e640 1 -- 172.21.15.52:0/2753319823 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f85240678c0 msgr2=0x7f8524069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.850+0000 7f854723e640 1 --2- 172.21.15.52:0/2753319823 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f85240678c0 0x7f8524069d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f852c006aa0 tx=0x7f852c008040 comp rx=0 tx=0).stop 2024-05-04T12:18:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.850+0000 7f854723e640 1 -- 172.21.15.52:0/2753319823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85300039c0 msgr2=0x7f854013d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.850+0000 7f854723e640 1 --2- 172.21.15.52:0/2753319823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85300039c0 0x7f854013d520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f853401c8e0 tx=0x7f8530005840 comp rx=0 tx=0).stop 2024-05-04T12:18:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.850+0000 7f854723e640 1 -- 172.21.15.52:0/2753319823 shutdown_connections 2024-05-04T12:18:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.850+0000 7f854723e640 1 --2- 172.21.15.52:0/2753319823 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f85240678c0 0x7f8524069d80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.850+0000 7f854723e640 1 --2- 172.21.15.52:0/2753319823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f854015e7b0 0x7f854015ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.850+0000 7f854723e640 1 --2- 172.21.15.52:0/2753319823 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8540144240 0x7f8540145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.850+0000 7f854723e640 1 --2- 172.21.15.52:0/2753319823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85300039c0 0x7f854013d520 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:34.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.850+0000 7f854723e640 1 -- 172.21.15.52:0/2753319823 >> 172.21.15.52:0/2753319823 conn(0x7f8540070690 msgr2=0x7f85400d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:34.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.850+0000 7f854723e640 1 -- 172.21.15.52:0/2753319823 shutdown_connections 2024-05-04T12:18:34.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:34.850+0000 7f854723e640 1 -- 172.21.15.52:0/2753319823 wait complete. 2024-05-04T12:18:34.867 INFO:teuthology.run_tasks:Running task sequential... 2024-05-04T12:18:34.875 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-04T12:18:34.875 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-04T12:18:34.876 INFO:teuthology.task.print:Enabling mgr modules 2024-05-04T12:18:34.876 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-04T12:18:34.909 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-04T12:18:34.909 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-04T12:18:34.909 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-04T12:18:34.910 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-04T12:18:34.910 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-04T12:18:34.910 INFO:tasks.ceph_fuse:client.2 config is {} 2024-05-04T12:18:34.910 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-04T12:18:34.910 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-04T12:18:34.910 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-04T12:18:34.910 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-04T12:18:34.910 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-04T12:18:34.910 INFO:tasks.ceph_fuse:client.3 config is {} 2024-05-04T12:18:34.910 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-04T12:18:34.911 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-04T12:18:34.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:34.911 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-05-04T12:18:34.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:34.920 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2024-05-04T12:18:34.978 INFO:teuthology.orchestra.run.smithi111.stderr:Cannot find device "ceph-brx" 2024-05-04T12:18:34.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T12:18:34.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:34.980 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-05-04T12:18:34.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:34.987 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-05-04T12:18:35.045 INFO:teuthology.orchestra.run.smithi052.stderr:Cannot find device "ceph-brx" 2024-05-04T12:18:35.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T12:18:35.047 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-04T12:18:35.047 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-04T12:18:35.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a8d86640 1 Processor -- start 2024-05-04T12:18:35.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a8d86640 1 -- start start 2024-05-04T12:18:35.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a8d86640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f50a415bf50 0x7f50a415c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:35.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a8d86640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f50a40aa0c0 0x7f50a40aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:35.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a8d86640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f50a40ccb50 0x7f50a40cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:35.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a8d86640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f50a406d1b0 con 0x7f50a40ccb50 2024-05-04T12:18:35.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a8d86640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f50a406d490 con 0x7f50a40aa0c0 2024-05-04T12:18:35.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a8d86640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f50a406d770 con 0x7f50a415bf50 2024-05-04T12:18:35.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a2d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f50a40ccb50 0x7f50a40cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:35.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a2d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f50a40ccb50 0x7f50a40cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53924/0 (socket says 172.21.15.52:53924) 2024-05-04T12:18:35.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a2d9d640 1 -- 172.21.15.52:0/1830782467 learned_addr learned my addr 172.21.15.52:0/1830782467 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:35.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a259c640 1 --2- 172.21.15.52:0/1830782467 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f50a415bf50 0x7f50a415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:35.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f509bfff640 1 --2- 172.21.15.52:0/1830782467 >> v2:172.21.15.111:3300/0 conn(0x7f50a40aa0c0 0x7f50a40aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:35.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a2d9d640 1 -- 172.21.15.52:0/1830782467 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f50a415bf50 msgr2=0x7f50a415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:35.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a2d9d640 1 --2- 172.21.15.52:0/1830782467 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f50a415bf50 0x7f50a415c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:35.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a2d9d640 1 -- 172.21.15.52:0/1830782467 >> v2:172.21.15.111:3300/0 conn(0x7f50a40aa0c0 msgr2=0x7f50a40aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:35.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a2d9d640 1 --2- 172.21.15.52:0/1830782467 >> v2:172.21.15.111:3300/0 conn(0x7f50a40aa0c0 0x7f50a40aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:35.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.222+0000 7f50a2d9d640 1 -- 172.21.15.52:0/1830782467 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a406da50 con 0x7f50a40ccb50 2024-05-04T12:18:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a2d9d640 1 --2- 172.21.15.52:0/1830782467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40ccb50 0x7f50a40cd000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f50a4069210 tx=0x7f509c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1886f4922664ffd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a1d9b640 1 -- 172.21.15.52:0/1830782467 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c0a5780 con 0x7f50a40ccb50 2024-05-04T12:18:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a1d9b640 1 -- 172.21.15.52:0/1830782467 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509c0a5920 con 0x7f50a40ccb50 2024-05-04T12:18:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a1d9b640 1 -- 172.21.15.52:0/1830782467 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c0a5cf0 con 0x7f50a40ccb50 2024-05-04T12:18:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 -- 172.21.15.52:0/1830782467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40ccb50 msgr2=0x7f50a40cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 --2- 172.21.15.52:0/1830782467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40ccb50 0x7f50a40cd000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f50a4069210 tx=0x7f509c01c9f0 comp rx=0 tx=0).stop 2024-05-04T12:18:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 -- 172.21.15.52:0/1830782467 shutdown_connections 2024-05-04T12:18:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 --2- 172.21.15.52:0/1830782467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40ccb50 0x7f50a40cd000 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 --2- 172.21.15.52:0/1830782467 >> v2:172.21.15.111:3300/0 conn(0x7f50a40aa0c0 0x7f50a40aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 --2- 172.21.15.52:0/1830782467 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f50a415bf50 0x7f50a415c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 -- 172.21.15.52:0/1830782467 >> 172.21.15.52:0/1830782467 conn(0x7f50a40b14a0 msgr2=0x7f50a40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 -- 172.21.15.52:0/1830782467 shutdown_connections 2024-05-04T12:18:35.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 -- 172.21.15.52:0/1830782467 wait complete. 2024-05-04T12:18:35.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 Processor -- start 2024-05-04T12:18:35.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 -- start start 2024-05-04T12:18:35.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40aa0c0 0x7f50a4147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:35.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50a40ccb50 0x7f50a4152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:35.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f50a415bf50 0x7f50a41526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:35.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f50a406ea60 con 0x7f50a40aa0c0 2024-05-04T12:18:35.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f50a406cf70 con 0x7f50a40ccb50 2024-05-04T12:18:35.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f50a406ebe0 con 0x7f50a415bf50 2024-05-04T12:18:35.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40aa0c0 0x7f50a4147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:35.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40aa0c0 0x7f50a4147aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53934/0 (socket says 172.21.15.52:53934) 2024-05-04T12:18:35.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a259c640 1 -- 172.21.15.52:0/1425707570 learned_addr learned my addr 172.21.15.52:0/1425707570 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:35.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a2d9d640 1 --2- 172.21.15.52:0/1425707570 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f50a415bf50 0x7f50a41526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:35.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f509bfff640 1 --2- 172.21.15.52:0/1425707570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50a40ccb50 0x7f50a4152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:35.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a259c640 1 -- 172.21.15.52:0/1425707570 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f50a415bf50 msgr2=0x7f50a41526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:35.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a259c640 1 --2- 172.21.15.52:0/1425707570 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f50a415bf50 0x7f50a41526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:35.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a259c640 1 -- 172.21.15.52:0/1425707570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50a40ccb50 msgr2=0x7f50a4152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:35.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a259c640 1 --2- 172.21.15.52:0/1425707570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50a40ccb50 0x7f50a4152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:35.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a259c640 1 -- 172.21.15.52:0/1425707570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a406eff0 con 0x7f50a40aa0c0 2024-05-04T12:18:35.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a259c640 1 --2- 172.21.15.52:0/1425707570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40aa0c0 0x7f50a4147aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f509400d5f0 tx=0x7f509400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:35.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f509affd640 1 -- 172.21.15.52:0/1425707570 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50940125f0 con 0x7f50a40aa0c0 2024-05-04T12:18:35.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f509affd640 1 -- 172.21.15.52:0/1425707570 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5094019070 con 0x7f50a40aa0c0 2024-05-04T12:18:35.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f509affd640 1 -- 172.21.15.52:0/1425707570 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50940129b0 con 0x7f50a40aa0c0 2024-05-04T12:18:35.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.226+0000 7f50a8d86640 1 -- 172.21.15.52:0/1425707570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a4154760 con 0x7f50a40aa0c0 2024-05-04T12:18:35.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.230+0000 7f50a8d86640 1 -- 172.21.15.52:0/1425707570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a4154ce0 con 0x7f50a40aa0c0 2024-05-04T12:18:35.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.230+0000 7f50a8d86640 1 -- 172.21.15.52:0/1425707570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a4062e50 con 0x7f50a40aa0c0 2024-05-04T12:18:35.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.230+0000 7f509affd640 1 -- 172.21.15.52:0/1425707570 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5094012b50 con 0x7f50a40aa0c0 2024-05-04T12:18:35.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.230+0000 7f509affd640 1 --2- 172.21.15.52:0/1425707570 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f50840678c0 0x7f5084069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:35.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.230+0000 7f509affd640 1 -- 172.21.15.52:0/1425707570 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5094086040 con 0x7f50a40aa0c0 2024-05-04T12:18:35.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.230+0000 7f509bfff640 1 --2- 172.21.15.52:0/1425707570 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f50840678c0 0x7f5084069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:35.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.230+0000 7f509bfff640 1 --2- 172.21.15.52:0/1425707570 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f50840678c0 0x7f5084069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f50a406f2f0 tx=0x7f508c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:35.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.234+0000 7f509affd640 1 -- 172.21.15.52:0/1425707570 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5094056c10 con 0x7f50a40aa0c0 2024-05-04T12:18:35.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.434+0000 7f50a8d86640 1 -- 172.21.15.52:0/1425707570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f50a4075320 con 0x7f50a40aa0c0 2024-05-04T12:18:35.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.434+0000 7f509affd640 1 -- 172.21.15.52:0/1425707570 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f5094056510 con 0x7f50a40aa0c0 2024-05-04T12:18:35.437 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-04T12:18:35.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.438+0000 7f50a8d86640 1 -- 172.21.15.52:0/1425707570 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f50840678c0 msgr2=0x7f5084069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:35.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.438+0000 7f50a8d86640 1 --2- 172.21.15.52:0/1425707570 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f50840678c0 0x7f5084069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f50a406f2f0 tx=0x7f508c008040 comp rx=0 tx=0).stop 2024-05-04T12:18:35.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.438+0000 7f50a8d86640 1 -- 172.21.15.52:0/1425707570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40aa0c0 msgr2=0x7f50a4147aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:35.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.438+0000 7f50a8d86640 1 --2- 172.21.15.52:0/1425707570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40aa0c0 0x7f50a4147aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f509400d5f0 tx=0x7f509400dab0 comp rx=0 tx=0).stop 2024-05-04T12:18:35.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.438+0000 7f50a8d86640 1 -- 172.21.15.52:0/1425707570 shutdown_connections 2024-05-04T12:18:35.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.438+0000 7f50a8d86640 1 --2- 172.21.15.52:0/1425707570 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f50840678c0 0x7f5084069d80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:35.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.438+0000 7f50a8d86640 1 --2- 172.21.15.52:0/1425707570 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f50a415bf50 0x7f50a41526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:35.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.438+0000 7f50a8d86640 1 --2- 172.21.15.52:0/1425707570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50a40ccb50 0x7f50a4152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:35.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.438+0000 7f50a8d86640 1 --2- 172.21.15.52:0/1425707570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40aa0c0 0x7f50a4147aa0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:35.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.438+0000 7f50a8d86640 1 -- 172.21.15.52:0/1425707570 >> 172.21.15.52:0/1425707570 conn(0x7f50a40b14a0 msgr2=0x7f50a40ce050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:35.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.438+0000 7f50a8d86640 1 -- 172.21.15.52:0/1425707570 shutdown_connections 2024-05-04T12:18:35.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35.438+0000 7f50a8d86640 1 -- 172.21.15.52:0/1425707570 wait complete. 2024-05-04T12:18:35.457 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-04T12:18:35.457 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-04T12:18:35.457 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-05-04T12:18:35.457 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-04T12:18:35.457 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-04T12:18:35.458 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-04T12:18:35.458 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-04T12:18:35.458 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-04T12:18:35.458 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-04T12:18:35.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:35.458 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-05-04T12:18:35.464 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-04T12:18:35.464 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-04T12:18:35.464 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-05-04T12:18:35.464 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:35.464 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-05-04T12:18:35.464 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:35.464 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:18:35.464 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:35.464 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-05-04T12:18:35.464 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42500sec preferred_lft 42500sec 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:41:0c:ed:b4 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-04T12:18:35.465 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:35.466 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-04T12:18:35.466 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:35.466 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-04T12:18:35.466 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add name ceph-brx type bridge 2024-05-04T12:18:35.466 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr flush dev ceph-brx 2024-05-04T12:18:35.466 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set ceph-brx up 2024-05-04T12:18:35.466 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-04T12:18:35.466 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-04T12:18:35.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:35.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:35.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:35.596 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-04T12:18:35.606 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-05-04T12:18:35.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:35.607 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-05-04T12:18:35.675 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-05-04T12:18:35.675 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-04T12:18:35.675 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-05-04T12:18:35.675 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-04T12:18:35.675 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-05-04T12:18:35.675 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-04T12:18:35.675 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:35.676 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-04T12:18:35.676 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-04T12:18:35.676 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-04T12:18:35.676 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-04T12:18:35.676 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-04T12:18:35.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:35.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:35.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:35.805 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-05-04T12:18:35.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:35.811 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-05-04T12:18:35.859 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:35.859 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-04T12:18:35.860 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:18:35.860 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-04T12:18:35.860 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-04T12:18:35.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:35.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:35.979 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-04T12:18:35.980 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:35.980 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-04T12:18:35.980 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-04T12:18:35.980 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-04T12:18:35.980 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-04T12:18:35.980 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-04T12:18:35.980 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-04T12:18:35.980 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-04T12:18:36.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:36.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:36.084 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:36.084 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-04T12:18:36.084 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-05-04T12:18:36.084 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.0 master ceph-brx 2024-05-04T12:18:36.084 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-04T12:18:36.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:36.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:36.188 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-04T12:18:36.188 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-04T12:18:36.188 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-04T12:18:36.235 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-04T12:18:36.235 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-04T12:18:36.235 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-04T12:18:36.283 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-04T12:18:36.341 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:18:36.388 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-04T12:18:36.388 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-04T12:18:36.388 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/run/lock 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-04T12:18:36.389 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/fs/nfsd 2024-05-04T12:18:36.390 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-04T12:18:36.390 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-04T12:18:36.390 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:18:36.390 INFO:teuthology.orchestra.run.smithi052.stdout:/run/rpc_pipefs 2024-05-04T12:18:36.390 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-04T12:18:36.390 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-04T12:18:36.390 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-04T12:18:36.390 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-04T12:18:36.390 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-04T12:18:36.390 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-04T12:18:36.390 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-04T12:18:36.390 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:18:36.390 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:18:36.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:36.391 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-04T12:18:36.439 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-04T12:18:36.439 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-04T12:18:36.484 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-04T12:18:36.500 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:18:36.551 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-04T12:18:36.552 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-04T12:18:36.552 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-04T12:18:36.552 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-04T12:18:36.552 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-04T12:18:36.552 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-04T12:18:36.552 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-04T12:18:36.552 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-04T12:18:36.552 INFO:teuthology.orchestra.run.smithi052.stdout:/run/lock 2024-05-04T12:18:36.552 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-04T12:18:36.552 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-04T12:18:36.552 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-04T12:18:36.552 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:18:36.552 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-04T12:18:36.552 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/fs/nfsd 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/run/rpc_pipefs 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:18:36.553 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:18:36.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:36.554 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-04T12:18:36.569 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:2024-05-04T12:18:36.566+0000 7fcd50c07580 -1 init, newargv = 0x55a2d6fb3b90 newargc=15 2024-05-04T12:18:36.569 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[34463]: starting ceph client 2024-05-04T12:18:36.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[34463]: starting fuse 2024-05-04T12:18:37.675 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-04T12:18:37.692 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:18:37.742 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-04T12:18:37.742 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-04T12:18:37.742 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-04T12:18:37.742 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-04T12:18:37.742 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-04T12:18:37.742 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-04T12:18:37.742 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-04T12:18:37.742 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-04T12:18:37.742 INFO:teuthology.orchestra.run.smithi052.stdout:/run/lock 2024-05-04T12:18:37.742 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-04T12:18:37.742 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-04T12:18:37.742 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-04T12:18:37.742 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:18:37.742 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/fs/nfsd 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/run/rpc_pipefs 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:18:37.743 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:18:37.744 INFO:teuthology.orchestra.run.smithi052.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-04T12:18:37.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:37.744 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-04T12:18:37.791 INFO:teuthology.orchestra.run.smithi052.stdout:45 2024-05-04T12:18:37.792 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-04T12:18:37.792 DEBUG:teuthology.orchestra.run.smithi052:> sudo stdin-killer -- python3 -c ' 2024-05-04T12:18:37.792 DEBUG:teuthology.orchestra.run.smithi052:> import glob 2024-05-04T12:18:37.792 DEBUG:teuthology.orchestra.run.smithi052:> import re 2024-05-04T12:18:37.792 DEBUG:teuthology.orchestra.run.smithi052:> import os 2024-05-04T12:18:37.792 DEBUG:teuthology.orchestra.run.smithi052:> import subprocess 2024-05-04T12:18:37.792 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-04T12:18:37.792 DEBUG:teuthology.orchestra.run.smithi052:> def _find_admin_socket(client_name): 2024-05-04T12:18:37.792 DEBUG:teuthology.orchestra.run.smithi052:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-04T12:18:37.792 DEBUG:teuthology.orchestra.run.smithi052:> files = glob.glob(asok_path) 2024-05-04T12:18:37.792 DEBUG:teuthology.orchestra.run.smithi052:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-04T12:18:37.792 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-04T12:18:37.792 DEBUG:teuthology.orchestra.run.smithi052:> # Given a non-glob path, it better be there 2024-05-04T12:18:37.792 DEBUG:teuthology.orchestra.run.smithi052:> if "*" not in asok_path: 2024-05-04T12:18:37.792 DEBUG:teuthology.orchestra.run.smithi052:> assert(len(files) == 1) 2024-05-04T12:18:37.793 DEBUG:teuthology.orchestra.run.smithi052:> return files[0] 2024-05-04T12:18:37.793 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-04T12:18:37.793 DEBUG:teuthology.orchestra.run.smithi052:> for f in files: 2024-05-04T12:18:37.793 DEBUG:teuthology.orchestra.run.smithi052:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-04T12:18:37.793 DEBUG:teuthology.orchestra.run.smithi052:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-04T12:18:37.793 DEBUG:teuthology.orchestra.run.smithi052:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-04T12:18:37.793 DEBUG:teuthology.orchestra.run.smithi052:> contents = proc_f.read() 2024-05-04T12:18:37.793 DEBUG:teuthology.orchestra.run.smithi052:> if mountpoint in contents: 2024-05-04T12:18:37.793 DEBUG:teuthology.orchestra.run.smithi052:> return f 2024-05-04T12:18:37.793 DEBUG:teuthology.orchestra.run.smithi052:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-04T12:18:37.793 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-04T12:18:37.793 DEBUG:teuthology.orchestra.run.smithi052:> print(_find_admin_socket("client.0")) 2024-05-04T12:18:37.793 DEBUG:teuthology.orchestra.run.smithi052:> ' 2024-05-04T12:18:37.922 INFO:teuthology.orchestra.run.smithi052.stdout:/var/run/ceph/ceph-client.0.34463.asok 2024-05-04T12:18:37.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:37.936 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.34463.asok 2024-05-04T12:18:37.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:37.936 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.34463.asok status 2024-05-04T12:18:38.096 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-04T12:18:38.096 INFO:teuthology.orchestra.run.smithi052.stdout: "metadata": { 2024-05-04T12:18:38.096 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_sha1": "09dbd6bba52d19ba471c33bc15c008e6ad158ea6", 2024-05-04T12:18:38.096 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-2455-g09dbd6bb (09dbd6bba52d19ba471c33bc15c008e6ad158ea6) squid (dev)", 2024-05-04T12:18:38.096 INFO:teuthology.orchestra.run.smithi052.stdout: "entity_id": "0", 2024-05-04T12:18:38.096 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": "smithi052", 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: "pid": "34463", 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: "root": "/" 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_count": 0, 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_pinned_count": 0, 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 4617, 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: "inst": { 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: "name": { 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "client", 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: "num": 4617 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-05-04T12:18:38.097 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2693806270 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2693806270 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: "inst_str": "client.4617 192.168.0.1:0/2693806270", 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: "addr_str": "192.168.0.1:0/2693806270", 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: "inode_count": 1, 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_epoch": 7, 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch": 21, 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch_barrier": 0, 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: "blocklisted": false, 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs" 2024-05-04T12:18:38.098 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-04T12:18:38.110 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-04T12:18:38.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.230+0000 7fa3c2df8640 1 Processor -- start 2024-05-04T12:18:38.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.230+0000 7fa3c2df8640 1 -- start start 2024-05-04T12:18:38.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3c2df8640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3bc067070 0x7fa3bc0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:38.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3c2df8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa3bc0ceff0 0x7fa3bc15e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:38.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3c2df8640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3bc0cb650 con 0x7fa3bc0cec10 2024-05-04T12:18:38.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3c2df8640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fa3bc0cb930 con 0x7fa3bc0ceff0 2024-05-04T12:18:38.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3c2df8640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa3bc0cbc10 con 0x7fa3bc067070 2024-05-04T12:18:38.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3c0b94640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa3bc0cec10 0x7fa3bc0d0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54380/0 (socket says 172.21.15.52:54380) 2024-05-04T12:18:38.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3c0b94640 1 -- 172.21.15.52:0/1398314437 learned_addr learned my addr 172.21.15.52:0/1398314437 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:38.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3c1395640 1 --2- 172.21.15.52:0/1398314437 >> v2:172.21.15.111:3300/0 conn(0x7fa3bc0ceff0 0x7fa3bc15e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:38.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bbfff640 1 --2- 172.21.15.52:0/1398314437 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3bc067070 0x7fa3bc0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:38.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 -- 172.21.15.52:0/1398314437 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 711847641 0 0) 0x7fa3bc0cb650 con 0x7fa3bc0cec10 2024-05-04T12:18:38.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 -- 172.21.15.52:0/1398314437 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3ac003530 con 0x7fa3bc0cec10 2024-05-04T12:18:38.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 -- 172.21.15.52:0/1398314437 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2529615389 0 0) 0x7fa3ac003530 con 0x7fa3bc0cec10 2024-05-04T12:18:38.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 -- 172.21.15.52:0/1398314437 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3bc067070 msgr2=0x7fa3bc0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:38.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 --2- 172.21.15.52:0/1398314437 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3bc067070 0x7fa3bc0ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:38.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 -- 172.21.15.52:0/1398314437 >> v2:172.21.15.111:3300/0 conn(0x7fa3bc0ceff0 msgr2=0x7fa3bc15e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:38.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 --2- 172.21.15.52:0/1398314437 >> v2:172.21.15.111:3300/0 conn(0x7fa3bc0ceff0 0x7fa3bc15e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:38.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 -- 172.21.15.52:0/1398314437 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3bc0b0570 con 0x7fa3bc0cec10 2024-05-04T12:18:38.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 -- 172.21.15.52:0/1398314437 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7fa3b0001680 con 0x7fa3bc0cec10 2024-05-04T12:18:38.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 -- 172.21.15.52:0/1398314437 >> v1:172.21.15.52:6789/0 conn(0x7fa3bc0cec10 legacy=0x7fa3bc0d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:38.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 --2- 172.21.15.52:0/1398314437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3ac0039c0 0x7fa3ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:38.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 --2- 172.21.15.52:0/1398314437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ac004810 0x7fa3ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:38.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 --2- 172.21.15.52:0/1398314437 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3bc067070 0x7fa3ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:38.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 -- 172.21.15.52:0/1398314437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc0cb930 con 0x7fa3ac004810 2024-05-04T12:18:38.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 -- 172.21.15.52:0/1398314437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc0cbc10 con 0x7fa3ac0039c0 2024-05-04T12:18:38.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bb7fe640 1 -- 172.21.15.52:0/1398314437 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ac00b9d0 con 0x7fa3bc067070 2024-05-04T12:18:38.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bbfff640 1 --2- 172.21.15.52:0/1398314437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ac004810 0x7fa3ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:38.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3c0b94640 1 --2- 172.21.15.52:0/1398314437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3ac0039c0 0x7fa3ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:38.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3c1395640 1 --2- 172.21.15.52:0/1398314437 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3bc067070 0x7fa3ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:38.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bbfff640 1 -- 172.21.15.52:0/1398314437 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3bc067070 msgr2=0x7fa3ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:38.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bbfff640 1 --2- 172.21.15.52:0/1398314437 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3bc067070 0x7fa3ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:38.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bbfff640 1 -- 172.21.15.52:0/1398314437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3ac0039c0 msgr2=0x7fa3ac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:38.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bbfff640 1 --2- 172.21.15.52:0/1398314437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3ac0039c0 0x7fa3ac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:38.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bbfff640 1 -- 172.21.15.52:0/1398314437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac003530 con 0x7fa3ac004810 2024-05-04T12:18:38.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.234+0000 7fa3bbfff640 1 --2- 172.21.15.52:0/1398314437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ac004810 0x7fa3ac004c90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8009ae0 tx=0x7fa3a801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3121a84a6cd97abb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:38.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3bb7fe640 1 -- 172.21.15.52:0/1398314437 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a80096b0 con 0x7fa3ac004810 2024-05-04T12:18:38.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3bb7fe640 1 -- 172.21.15.52:0/1398314437 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a80a6480 con 0x7fa3ac004810 2024-05-04T12:18:38.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3bb7fe640 1 -- 172.21.15.52:0/1398314437 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a80a6870 con 0x7fa3ac004810 2024-05-04T12:18:38.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c2df8640 1 -- 172.21.15.52:0/1398314437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ac004810 msgr2=0x7fa3ac004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:38.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c2df8640 1 --2- 172.21.15.52:0/1398314437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ac004810 0x7fa3ac004c90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8009ae0 tx=0x7fa3a801ca70 comp rx=0 tx=0).stop 2024-05-04T12:18:38.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c1395640 1 -- 172.21.15.52:0/1398314437 reap_dead start 2024-05-04T12:18:38.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c2df8640 1 -- 172.21.15.52:0/1398314437 shutdown_connections 2024-05-04T12:18:38.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c2df8640 1 -- 172.21.15.52:0/1398314437 >> 172.21.15.52:0/1398314437 conn(0x7fa3bc0cd5b0 msgr2=0x7fa3bc0cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:38.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c2df8640 1 -- 172.21.15.52:0/1398314437 shutdown_connections 2024-05-04T12:18:38.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c2df8640 1 -- 172.21.15.52:0/1398314437 wait complete. 2024-05-04T12:18:38.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c2df8640 1 Processor -- start 2024-05-04T12:18:38.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c2df8640 1 -- start start 2024-05-04T12:18:38.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c2df8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ac004810 0x7fa3bc1577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:38.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c2df8640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3bc1539b0 0x7fa3bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:38.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c2df8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3bc1530d0 0x7fa3bc158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:38.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c2df8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc0adc20 con 0x7fa3ac004810 2024-05-04T12:18:38.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c2df8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc0adaa0 con 0x7fa3bc1530d0 2024-05-04T12:18:38.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c2df8640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa3bc15e5a0 con 0x7fa3bc1539b0 2024-05-04T12:18:38.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c0b94640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ac004810 0x7fa3bc1577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:38.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c0b94640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ac004810 0x7fa3bc1577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36046/0 (socket says 172.21.15.52:36046) 2024-05-04T12:18:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c0b94640 1 -- 172.21.15.52:0/259581346 learned_addr learned my addr 172.21.15.52:0/259581346 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3c1395640 1 --2- 172.21.15.52:0/259581346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3bc1530d0 0x7fa3bc158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.238+0000 7fa3bbfff640 1 --2- 172.21.15.52:0/259581346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3bc1539b0 0x7fa3bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3c0b94640 1 -- 172.21.15.52:0/259581346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3bc1539b0 msgr2=0x7fa3bc06ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3c0b94640 1 --2- 172.21.15.52:0/259581346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3bc1539b0 0x7fa3bc06ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3c0b94640 1 -- 172.21.15.52:0/259581346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3bc1530d0 msgr2=0x7fa3bc158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3c0b94640 1 --2- 172.21.15.52:0/259581346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3bc1530d0 0x7fa3bc158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3c0b94640 1 -- 172.21.15.52:0/259581346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3bc06dd40 con 0x7fa3ac004810 2024-05-04T12:18:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3c0b94640 1 --2- 172.21.15.52:0/259581346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ac004810 0x7fa3bc1577d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0000fc0 tx=0x7fa3b0011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3b97fa640 1 -- 172.21.15.52:0/259581346 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0010040 con 0x7fa3ac004810 2024-05-04T12:18:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3b97fa640 1 -- 172.21.15.52:0/259581346 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b001a070 con 0x7fa3ac004810 2024-05-04T12:18:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3c2df8640 1 -- 172.21.15.52:0/259581346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3bc06e020 con 0x7fa3ac004810 2024-05-04T12:18:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3b97fa640 1 -- 172.21.15.52:0/259581346 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000fd90 con 0x7fa3ac004810 2024-05-04T12:18:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3c2df8640 1 -- 172.21.15.52:0/259581346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3bc06e5e0 con 0x7fa3ac004810 2024-05-04T12:18:38.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3c2df8640 1 -- 172.21.15.52:0/259581346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3bc062e50 con 0x7fa3ac004810 2024-05-04T12:18:38.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3b97fa640 1 -- 172.21.15.52:0/259581346 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa3b001f460 con 0x7fa3ac004810 2024-05-04T12:18:38.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3b97fa640 1 --2- 172.21.15.52:0/259581346 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa3a00678f0 0x7fa3a0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:38.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3b97fa640 1 -- 172.21.15.52:0/259581346 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa3b0088d80 con 0x7fa3ac004810 2024-05-04T12:18:38.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3bbfff640 1 --2- 172.21.15.52:0/259581346 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa3a00678f0 0x7fa3a0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:38.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.242+0000 7fa3bbfff640 1 --2- 172.21.15.52:0/259581346 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa3a00678f0 0x7fa3a0069db0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80ad000 tx=0x7fa3a80b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:38.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.246+0000 7fa3b97fa640 1 -- 172.21.15.52:0/259581346 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa3b0059950 con 0x7fa3ac004810 2024-05-04T12:18:38.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.446+0000 7fa3c2df8640 1 -- 172.21.15.52:0/259581346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa3bc0b1c30 con 0x7fa3ac004810 2024-05-04T12:18:38.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.446+0000 7fa3b97fa640 1 -- 172.21.15.52:0/259581346 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fa3b0059250 con 0x7fa3ac004810 2024-05-04T12:18:38.448 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-04T12:18:38.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.446+0000 7fa3c2df8640 1 -- 172.21.15.52:0/259581346 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa3a00678f0 msgr2=0x7fa3a0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:38.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.446+0000 7fa3c2df8640 1 --2- 172.21.15.52:0/259581346 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa3a00678f0 0x7fa3a0069db0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80ad000 tx=0x7fa3a80b2040 comp rx=0 tx=0).stop 2024-05-04T12:18:38.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.446+0000 7fa3c2df8640 1 -- 172.21.15.52:0/259581346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ac004810 msgr2=0x7fa3bc1577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:38.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.446+0000 7fa3c2df8640 1 --2- 172.21.15.52:0/259581346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ac004810 0x7fa3bc1577d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0000fc0 tx=0x7fa3b0011ca0 comp rx=0 tx=0).stop 2024-05-04T12:18:38.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.446+0000 7fa3c2df8640 1 -- 172.21.15.52:0/259581346 shutdown_connections 2024-05-04T12:18:38.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.446+0000 7fa3c2df8640 1 --2- 172.21.15.52:0/259581346 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa3a00678f0 0x7fa3a0069db0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:38.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.446+0000 7fa3c2df8640 1 --2- 172.21.15.52:0/259581346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3bc1530d0 0x7fa3bc158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:38.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.446+0000 7fa3c2df8640 1 --2- 172.21.15.52:0/259581346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3bc1539b0 0x7fa3bc06ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:38.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.446+0000 7fa3c2df8640 1 --2- 172.21.15.52:0/259581346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ac004810 0x7fa3bc1577d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:38.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.446+0000 7fa3c2df8640 1 -- 172.21.15.52:0/259581346 >> 172.21.15.52:0/259581346 conn(0x7fa3bc0cd5b0 msgr2=0x7fa3bc0ae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:38.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.446+0000 7fa3c2df8640 1 -- 172.21.15.52:0/259581346 shutdown_connections 2024-05-04T12:18:38.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38.446+0000 7fa3c2df8640 1 -- 172.21.15.52:0/259581346 wait complete. 2024-05-04T12:18:38.465 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-04T12:18:38.466 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-04T12:18:38.466 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-05-04T12:18:38.466 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-04T12:18:38.466 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-04T12:18:38.466 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-04T12:18:38.466 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-04T12:18:38.466 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-04T12:18:38.466 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-04T12:18:38.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:38.466 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-05-04T12:18:38.471 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-04T12:18:38.471 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-04T12:18:38.471 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42497sec preferred_lft 42497sec 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-05-04T12:18:38.472 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:41:0c:ed:b4 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:18:38.473 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-04T12:18:38.474 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:38.474 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-04T12:18:38.524 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-05-04T12:18:38.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:38.526 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-05-04T12:18:38.571 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-05-04T12:18:38.571 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-04T12:18:38.571 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-05-04T12:18:38.571 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-04T12:18:38.571 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-05-04T12:18:38.571 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-04T12:18:38.571 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:38.572 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-04T12:18:38.572 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-04T12:18:38.572 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-04T12:18:38.572 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-04T12:18:38.572 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-04T12:18:38.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:38.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:38.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:38.683 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-05-04T12:18:38.731 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-04T12:18:38.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:38.732 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-05-04T12:18:38.781 INFO:teuthology.orchestra.run.smithi052.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-04T12:18:38.782 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:38.782 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-04T12:18:38.782 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T12:18:38.782 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-04T12:18:38.782 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-04T12:18:38.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:38.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:38.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:38.898 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-04T12:18:38.908 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-04T12:18:38.909 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-04T12:18:38.909 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-05-04T12:18:38.909 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.909 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-05-04T12:18:38.909 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.909 INFO:teuthology.orchestra.run.smithi052.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-04T12:18:38.909 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 1e:21:e4:ab:d2:36 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-04T12:18:38.909 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-04T12:18:38.909 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.909 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::1c21:e4ff:feab:d236/64 scope link 2024-05-04T12:18:38.909 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:38.910 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-04T12:18:38.964 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-04T12:18:38.964 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-04T12:18:38.964 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-05-04T12:18:38.965 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.965 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-05-04T12:18:38.965 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.965 INFO:teuthology.orchestra.run.smithi052.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-04T12:18:38.965 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 1e:21:e4:ab:d2:36 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-04T12:18:38.965 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-04T12:18:38.965 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.965 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::1c21:e4ff:feab:d236/64 scope link 2024-05-04T12:18:38.965 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:38.966 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-04T12:18:38.966 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:38.966 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-04T12:18:38.966 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-04T12:18:38.966 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-04T12:18:38.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-04T12:18:38.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-04T12:18:38.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-04T12:18:38.967 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-04T12:18:39.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:39.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:39.113 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:39.113 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-04T12:18:39.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.1 up 2024-05-04T12:18:39.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.1 master ceph-brx 2024-05-04T12:18:39.113 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-04T12:18:39.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:39.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:39.179 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-05-04T12:18:39.179 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-04T12:18:39.179 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-04T12:18:39.226 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-04T12:18:39.227 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-04T12:18:39.227 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-04T12:18:39.278 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-04T12:18:39.335 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:18:39.384 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-04T12:18:39.384 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-04T12:18:39.384 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-04T12:18:39.384 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-04T12:18:39.384 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-04T12:18:39.384 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-04T12:18:39.384 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-04T12:18:39.384 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-04T12:18:39.384 INFO:teuthology.orchestra.run.smithi052.stdout:/run/lock 2024-05-04T12:18:39.384 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-04T12:18:39.384 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-04T12:18:39.384 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/fs/nfsd 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/run/rpc_pipefs 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-04T12:18:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-04T12:18:39.386 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-04T12:18:39.386 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:18:39.386 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:18:39.386 INFO:teuthology.orchestra.run.smithi052.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-04T12:18:39.386 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T12:18:39.386 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T12:18:39.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:39.387 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-04T12:18:39.435 INFO:teuthology.orchestra.run.smithi052.stdout:45 2024-05-04T12:18:39.435 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-05-04T12:18:39.435 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-05-04T12:18:39.480 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-04T12:18:39.493 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:18:39.539 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-04T12:18:39.539 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-04T12:18:39.539 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-04T12:18:39.539 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-04T12:18:39.539 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-04T12:18:39.539 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-04T12:18:39.539 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-04T12:18:39.539 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-04T12:18:39.539 INFO:teuthology.orchestra.run.smithi052.stdout:/run/lock 2024-05-04T12:18:39.539 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-04T12:18:39.539 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-04T12:18:39.539 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/fs/nfsd 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/run/rpc_pipefs 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-04T12:18:39.540 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-04T12:18:39.541 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-04T12:18:39.541 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-04T12:18:39.541 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:18:39.541 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:18:39.541 INFO:teuthology.orchestra.run.smithi052.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-04T12:18:39.541 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T12:18:39.541 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T12:18:39.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:39.542 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-04T12:18:39.564 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi052.stderr:2024-05-04T12:18:39.562+0000 7f4a87556580 -1 init, newargv = 0x5649078a43a0 newargc=15 2024-05-04T12:18:39.564 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi052.stderr:ceph-fuse[34608]: starting ceph client 2024-05-04T12:18:39.568 INFO:teuthology.orchestra.run.smithi052.stdout:45 2024-05-04T12:18:39.571 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi052.stderr:ceph-fuse[34608]: starting fuse 2024-05-04T12:18:40.669 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-04T12:18:40.685 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:18:40.732 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-04T12:18:40.732 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-04T12:18:40.732 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-04T12:18:40.732 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-04T12:18:40.732 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-04T12:18:40.732 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/run/lock 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/fs/nfsd 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-04T12:18:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:18:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:/run/rpc_pipefs 2024-05-04T12:18:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-04T12:18:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-04T12:18:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-04T12:18:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-04T12:18:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-04T12:18:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-04T12:18:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-04T12:18:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:18:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:18:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-04T12:18:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T12:18:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T12:18:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:/home/ubuntu/cephtest/mnt.2 2024-05-04T12:18:40.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:40.735 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-04T12:18:40.783 INFO:teuthology.orchestra.run.smithi052.stdout:45 2024-05-04T12:18:40.783 INFO:teuthology.orchestra.run.smithi052.stdout:46 2024-05-04T12:18:40.784 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-05-04T12:18:40.784 DEBUG:teuthology.orchestra.run.smithi052:> sudo stdin-killer -- python3 -c ' 2024-05-04T12:18:40.784 DEBUG:teuthology.orchestra.run.smithi052:> import glob 2024-05-04T12:18:40.784 DEBUG:teuthology.orchestra.run.smithi052:> import re 2024-05-04T12:18:40.784 DEBUG:teuthology.orchestra.run.smithi052:> import os 2024-05-04T12:18:40.784 DEBUG:teuthology.orchestra.run.smithi052:> import subprocess 2024-05-04T12:18:40.784 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-04T12:18:40.784 DEBUG:teuthology.orchestra.run.smithi052:> def _find_admin_socket(client_name): 2024-05-04T12:18:40.784 DEBUG:teuthology.orchestra.run.smithi052:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-05-04T12:18:40.784 DEBUG:teuthology.orchestra.run.smithi052:> files = glob.glob(asok_path) 2024-05-04T12:18:40.784 DEBUG:teuthology.orchestra.run.smithi052:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-05-04T12:18:40.784 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-04T12:18:40.784 DEBUG:teuthology.orchestra.run.smithi052:> # Given a non-glob path, it better be there 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> if "*" not in asok_path: 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> assert(len(files) == 1) 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> return files[0] 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> for f in files: 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> contents = proc_f.read() 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> if mountpoint in contents: 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> return f 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> print(_find_admin_socket("client.2")) 2024-05-04T12:18:40.785 DEBUG:teuthology.orchestra.run.smithi052:> ' 2024-05-04T12:18:40.912 INFO:teuthology.orchestra.run.smithi052.stdout:/var/run/ceph/ceph-client.2.34608.asok 2024-05-04T12:18:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:40.925 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.34608.asok 2024-05-04T12:18:40.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:40.925 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.34608.asok status 2024-05-04T12:18:41.079 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-04T12:18:41.079 INFO:teuthology.orchestra.run.smithi052.stdout: "metadata": { 2024-05-04T12:18:41.079 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_sha1": "09dbd6bba52d19ba471c33bc15c008e6ad158ea6", 2024-05-04T12:18:41.079 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-2455-g09dbd6bb (09dbd6bba52d19ba471c33bc15c008e6ad158ea6) squid (dev)", 2024-05-04T12:18:41.079 INFO:teuthology.orchestra.run.smithi052.stdout: "entity_id": "2", 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": "smithi052", 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: "pid": "34608", 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: "root": "/" 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_count": 0, 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_pinned_count": 0, 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 4629, 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: "inst": { 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: "name": { 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "client", 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: "num": 4629 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-05-04T12:18:41.080 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.2:0", 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 3884097125 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.2:0", 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 3884097125 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: "inst_str": "client.4629 192.168.0.2:0/3884097125", 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: "addr_str": "192.168.0.2:0/3884097125", 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: "inode_count": 1, 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_epoch": 7, 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch": 21, 2024-05-04T12:18:41.081 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch_barrier": 0, 2024-05-04T12:18:41.082 INFO:teuthology.orchestra.run.smithi052.stdout: "blocklisted": false, 2024-05-04T12:18:41.082 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs" 2024-05-04T12:18:41.082 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-04T12:18:41.093 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-04T12:18:41.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.210+0000 7fe9f38c8640 1 Processor -- start 2024-05-04T12:18:41.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.210+0000 7fe9f38c8640 1 -- start start 2024-05-04T12:18:41.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f38c8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe9ec0cd190 0x7fe9ec0723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:41.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f38c8640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ec072aa0 con 0x7fe9ec15c7f0 2024-05-04T12:18:41.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f38c8640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ec0708e0 con 0x7fe9ec0ccdb0 2024-05-04T12:18:41.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f38c8640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fe9ec070690 con 0x7fe9ec0cd190 2024-05-04T12:18:41.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f0e63640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fe9ec15c7f0 0x7fe9ec06de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54384/0 (socket says 172.21.15.52:54384) 2024-05-04T12:18:41.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f0e63640 1 -- 172.21.15.52:0/745483253 learned_addr learned my addr 172.21.15.52:0/745483253 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:41.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f1e65640 1 --2- 172.21.15.52:0/745483253 >> v2:172.21.15.111:3300/0 conn(0x7fe9ec0cd190 0x7fe9ec0723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:41.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 -- 172.21.15.52:0/745483253 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2160540414 0 0) 0x7fe9ec072aa0 con 0x7fe9ec15c7f0 2024-05-04T12:18:41.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 -- 172.21.15.52:0/745483253 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9d8003530 con 0x7fe9ec15c7f0 2024-05-04T12:18:41.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 -- 172.21.15.52:0/745483253 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3610949204 0 0) 0x7fe9d8003530 con 0x7fe9ec15c7f0 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 -- 172.21.15.52:0/745483253 >> v2:172.21.15.111:3300/0 conn(0x7fe9ec0cd190 msgr2=0x7fe9ec0723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 --2- 172.21.15.52:0/745483253 >> v2:172.21.15.111:3300/0 conn(0x7fe9ec0cd190 0x7fe9ec0723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 -- 172.21.15.52:0/745483253 >> v1:172.21.15.111:6789/0 conn(0x7fe9ec0ccdb0 legacy=0x7fe9ec158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 -- 172.21.15.52:0/745483253 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec070bc0 con 0x7fe9ec15c7f0 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 -- 172.21.15.52:0/745483253 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7fe9dc001680 con 0x7fe9ec15c7f0 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 -- 172.21.15.52:0/745483253 >> v1:172.21.15.52:6789/0 conn(0x7fe9ec15c7f0 legacy=0x7fe9ec06de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 --2- 172.21.15.52:0/745483253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9d80039c0 0x7fe9d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 --2- 172.21.15.52:0/745483253 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9d8004810 0x7fe9d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 --2- 172.21.15.52:0/745483253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9d80051d0 0x7fe9d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 -- 172.21.15.52:0/745483253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec070690 con 0x7fe9d80051d0 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 -- 172.21.15.52:0/745483253 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d800c170 con 0x7fe9d80039c0 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9e3fff640 1 -- 172.21.15.52:0/745483253 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d800c370 con 0x7fe9d8004810 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f1e65640 1 --2- 172.21.15.52:0/745483253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9d80051d0 0x7fe9d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f0e63640 1 --2- 172.21.15.52:0/745483253 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9d8004810 0x7fe9d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f1664640 1 --2- 172.21.15.52:0/745483253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9d80039c0 0x7fe9d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:41.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f1e65640 1 -- 172.21.15.52:0/745483253 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9d8004810 msgr2=0x7fe9d8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:41.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f1e65640 1 --2- 172.21.15.52:0/745483253 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9d8004810 0x7fe9d8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:41.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f1e65640 1 -- 172.21.15.52:0/745483253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9d80039c0 msgr2=0x7fe9d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:41.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f1e65640 1 --2- 172.21.15.52:0/745483253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9d80039c0 0x7fe9d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:41.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f1e65640 1 -- 172.21.15.52:0/745483253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d8003530 con 0x7fe9d80051d0 2024-05-04T12:18:41.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.214+0000 7fe9f1e65640 1 -- 172.21.15.52:0/745483253 reap_dead start 2024-05-04T12:18:41.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f1e65640 1 --2- 172.21.15.52:0/745483253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9d80051d0 0x7fe9d800b9d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009fc0 tx=0x7fe9e401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5f83f380252f5f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:41.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9e3fff640 1 -- 172.21.15.52:0/745483253 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4009c10 con 0x7fe9d80051d0 2024-05-04T12:18:41.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9e3fff640 1 -- 172.21.15.52:0/745483253 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9e40a5c00 con 0x7fe9d80051d0 2024-05-04T12:18:41.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9e3fff640 1 -- 172.21.15.52:0/745483253 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40039a0 con 0x7fe9d80051d0 2024-05-04T12:18:41.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 -- 172.21.15.52:0/745483253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9d80051d0 msgr2=0x7fe9d800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:41.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 --2- 172.21.15.52:0/745483253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9d80051d0 0x7fe9d800b9d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009fc0 tx=0x7fe9e401ccc0 comp rx=0 tx=0).stop 2024-05-04T12:18:41.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 -- 172.21.15.52:0/745483253 shutdown_connections 2024-05-04T12:18:41.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 --2- 172.21.15.52:0/745483253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9d80051d0 0x7fe9d800b9d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:41.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 -- 172.21.15.52:0/745483253 >> 172.21.15.52:0/745483253 conn(0x7fe9ec0b14a0 msgr2=0x7fe9ec0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:41.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 -- 172.21.15.52:0/745483253 shutdown_connections 2024-05-04T12:18:41.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 -- 172.21.15.52:0/745483253 wait complete. 2024-05-04T12:18:41.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 Processor -- start 2024-05-04T12:18:41.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 -- start start 2024-05-04T12:18:41.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9d80051d0 0x7fe9ec0aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ec0b0370 0x7fe9ec06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9ec0b0f20 0x7fe9ec1496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec15c4b0 con 0x7fe9ec0b0370 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec15c330 con 0x7fe9d80051d0 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ec15c630 con 0x7fe9ec0b0f20 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f0e63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ec0b0370 0x7fe9ec06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f0e63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ec0b0370 0x7fe9ec06cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36070/0 (socket says 172.21.15.52:36070) 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f0e63640 1 -- 172.21.15.52:0/2138017460 learned_addr learned my addr 172.21.15.52:0/2138017460 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f1664640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9d80051d0 0x7fe9ec0aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f1e65640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9ec0b0f20 0x7fe9ec1496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f0e63640 1 -- 172.21.15.52:0/2138017460 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9ec0b0f20 msgr2=0x7fe9ec1496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f0e63640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9ec0b0f20 0x7fe9ec1496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f0e63640 1 -- 172.21.15.52:0/2138017460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9d80051d0 msgr2=0x7fe9ec0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f0e63640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9d80051d0 0x7fe9ec0aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f0e63640 1 -- 172.21.15.52:0/2138017460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec0af2d0 con 0x7fe9ec0b0370 2024-05-04T12:18:41.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f1664640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9d80051d0 0x7fe9ec0aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:41.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f0e63640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ec0b0370 0x7fe9ec06cfc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc011840 tx=0x7fe9dc011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:41.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9e1ffb640 1 -- 172.21.15.52:0/2138017460 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc010040 con 0x7fe9ec0b0370 2024-05-04T12:18:41.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9e1ffb640 1 -- 172.21.15.52:0/2138017460 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9dc01a070 con 0x7fe9ec0b0370 2024-05-04T12:18:41.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 -- 172.21.15.52:0/2138017460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ec0af5b0 con 0x7fe9ec0b0370 2024-05-04T12:18:41.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9e1ffb640 1 -- 172.21.15.52:0/2138017460 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc00fd60 con 0x7fe9ec0b0370 2024-05-04T12:18:41.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.218+0000 7fe9f38c8640 1 -- 172.21.15.52:0/2138017460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ec0afb70 con 0x7fe9ec0b0370 2024-05-04T12:18:41.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.222+0000 7fe9f38c8640 1 -- 172.21.15.52:0/2138017460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ec062e50 con 0x7fe9ec0b0370 2024-05-04T12:18:41.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.222+0000 7fe9e1ffb640 1 -- 172.21.15.52:0/2138017460 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe9dc01f460 con 0x7fe9ec0b0370 2024-05-04T12:18:41.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.222+0000 7fe9e1ffb640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe9d00678c0 0x7fe9d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:41.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.222+0000 7fe9e1ffb640 1 -- 172.21.15.52:0/2138017460 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe9dc089950 con 0x7fe9ec0b0370 2024-05-04T12:18:41.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.222+0000 7fe9f1664640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe9d00678c0 0x7fe9d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:41.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.222+0000 7fe9f1664640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe9d00678c0 0x7fe9d0069d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000c10 tx=0x7fe9e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:41.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.226+0000 7fe9e1ffb640 1 -- 172.21.15.52:0/2138017460 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe9dc05a500 con 0x7fe9ec0b0370 2024-05-04T12:18:41.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.422+0000 7fe9f38c8640 1 -- 172.21.15.52:0/2138017460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe9ec075320 con 0x7fe9ec0b0370 2024-05-04T12:18:41.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.426+0000 7fe9e1ffb640 1 -- 172.21.15.52:0/2138017460 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fe9dc059e00 con 0x7fe9ec0b0370 2024-05-04T12:18:41.426 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-04T12:18:41.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.426+0000 7fe9f38c8640 1 -- 172.21.15.52:0/2138017460 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe9d00678c0 msgr2=0x7fe9d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:41.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.426+0000 7fe9f38c8640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe9d00678c0 0x7fe9d0069d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000c10 tx=0x7fe9e8009040 comp rx=0 tx=0).stop 2024-05-04T12:18:41.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.426+0000 7fe9f38c8640 1 -- 172.21.15.52:0/2138017460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ec0b0370 msgr2=0x7fe9ec06cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:41.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.426+0000 7fe9f38c8640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ec0b0370 0x7fe9ec06cfc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc011840 tx=0x7fe9dc011d00 comp rx=0 tx=0).stop 2024-05-04T12:18:41.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.426+0000 7fe9f38c8640 1 -- 172.21.15.52:0/2138017460 shutdown_connections 2024-05-04T12:18:41.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.426+0000 7fe9f38c8640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe9d00678c0 0x7fe9d0069d80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:41.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.426+0000 7fe9f38c8640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9ec0b0f20 0x7fe9ec1496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:41.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.426+0000 7fe9f38c8640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ec0b0370 0x7fe9ec06cfc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:41.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.426+0000 7fe9f38c8640 1 --2- 172.21.15.52:0/2138017460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9d80051d0 0x7fe9ec0aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:41.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.426+0000 7fe9f38c8640 1 -- 172.21.15.52:0/2138017460 >> 172.21.15.52:0/2138017460 conn(0x7fe9ec0b14a0 msgr2=0x7fe9ec15e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:41.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.426+0000 7fe9f38c8640 1 -- 172.21.15.52:0/2138017460 shutdown_connections 2024-05-04T12:18:41.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:41.426+0000 7fe9f38c8640 1 -- 172.21.15.52:0/2138017460 wait complete. 2024-05-04T12:18:41.445 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-04T12:18:41.445 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-04T12:18:41.445 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2024-05-04T12:18:41.445 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-04T12:18:41.445 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-04T12:18:41.445 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-04T12:18:41.445 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-04T12:18:41.445 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-04T12:18:41.445 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-04T12:18:41.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:41.446 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2024-05-04T12:18:41.452 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-04T12:18:41.453 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-04T12:18:41.453 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-05-04T12:18:41.453 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:41.453 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-05-04T12:18:41.453 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:41.453 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:18:41.453 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:41.453 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2024-05-04T12:18:41.453 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:18:41.453 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:41.453 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2024-05-04T12:18:41.453 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:18:41.453 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:41.453 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-04T12:18:41.454 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:41.454 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-04T12:18:41.454 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42510sec preferred_lft 42510sec 2024-05-04T12:18:41.454 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2024-05-04T12:18:41.454 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:41.454 INFO:teuthology.orchestra.run.smithi111.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-04T12:18:41.454 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 02:42:7f:88:9e:58 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:41.454 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-04T12:18:41.454 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:41.454 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-04T12:18:41.454 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:41.454 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-04T12:18:41.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add name ceph-brx type bridge 2024-05-04T12:18:41.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr flush dev ceph-brx 2024-05-04T12:18:41.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set ceph-brx up 2024-05-04T12:18:41.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-04T12:18:41.455 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-04T12:18:41.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:41.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:41.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:41.587 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-04T12:18:41.596 INFO:teuthology.orchestra.run.smithi111.stdout:1 2024-05-04T12:18:41.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:41.598 DEBUG:teuthology.orchestra.run.smithi111:> route 2024-05-04T12:18:41.673 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2024-05-04T12:18:41.673 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-04T12:18:41.673 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-04T12:18:41.673 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-04T12:18:41.673 INFO:teuthology.orchestra.run.smithi111.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-04T12:18:41.673 INFO:teuthology.orchestra.run.smithi111.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-04T12:18:41.673 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-04T12:18:41.674 INFO:teuthology.orchestra.run.smithi111.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-04T12:18:41.674 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-04T12:18:41.674 INFO:teuthology.orchestra.run.smithi111.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-04T12:18:41.674 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-04T12:18:41.674 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:41.674 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-04T12:18:41.674 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-04T12:18:41.674 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-04T12:18:41.674 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-04T12:18:41.674 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-04T12:18:41.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:41.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:41.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:41.799 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-05-04T12:18:41.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:41.805 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2024-05-04T12:18:41.850 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:41.850 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-04T12:18:41.851 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T12:18:41.851 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-04T12:18:41.851 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-04T12:18:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:41.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:41.967 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-04T12:18:41.967 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:41.967 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-04T12:18:41.967 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-04T12:18:41.967 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-04T12:18:41.967 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-04T12:18:41.967 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-04T12:18:41.967 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-04T12:18:41.967 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-04T12:18:42.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:42.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:42.066 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:42.067 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-04T12:18:42.067 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2024-05-04T12:18:42.067 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.0 master ceph-brx 2024-05-04T12:18:42.067 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-04T12:18:42.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:42.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:42.170 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-04T12:18:42.170 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-04T12:18:42.171 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-04T12:18:42.217 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-04T12:18:42.217 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-04T12:18:42.217 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-04T12:18:42.265 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-05-04T12:18:42.322 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:18:42.371 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2024-05-04T12:18:42.371 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2024-05-04T12:18:42.371 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2024-05-04T12:18:42.371 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2024-05-04T12:18:42.371 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2024-05-04T12:18:42.371 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2024-05-04T12:18:42.371 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2024-05-04T12:18:42.371 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/run/lock 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/fs/nfsd 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/run/rpc_pipefs 2024-05-04T12:18:42.372 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2024-05-04T12:18:42.373 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2024-05-04T12:18:42.373 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-4 2024-05-04T12:18:42.373 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-5 2024-05-04T12:18:42.373 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-6 2024-05-04T12:18:42.373 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-7 2024-05-04T12:18:42.373 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2024-05-04T12:18:42.373 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T12:18:42.373 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T12:18:42.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:42.374 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-05-04T12:18:42.422 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-04T12:18:42.422 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-04T12:18:42.467 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-05-04T12:18:42.480 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:18:42.526 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2024-05-04T12:18:42.526 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2024-05-04T12:18:42.526 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2024-05-04T12:18:42.526 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2024-05-04T12:18:42.526 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/run/lock 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/fs/nfsd 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2024-05-04T12:18:42.527 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2024-05-04T12:18:42.528 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:18:42.528 INFO:teuthology.orchestra.run.smithi111.stdout:/run/rpc_pipefs 2024-05-04T12:18:42.528 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2024-05-04T12:18:42.528 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2024-05-04T12:18:42.528 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-4 2024-05-04T12:18:42.528 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-5 2024-05-04T12:18:42.528 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-6 2024-05-04T12:18:42.528 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-7 2024-05-04T12:18:42.528 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2024-05-04T12:18:42.528 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T12:18:42.528 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T12:18:42.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:42.529 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-05-04T12:18:42.544 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi111.stderr:2024-05-04T12:18:42.542+0000 7fcc2b689580 -1 init, newargv = 0x55cb14db6bf0 newargc=15 2024-05-04T12:18:42.544 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi111.stderr:ceph-fuse[33492]: starting ceph client 2024-05-04T12:18:42.550 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi111.stderr:ceph-fuse[33492]: starting fuse 2024-05-04T12:18:43.649 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-05-04T12:18:43.666 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:18:43.716 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2024-05-04T12:18:43.716 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2024-05-04T12:18:43.716 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2024-05-04T12:18:43.716 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2024-05-04T12:18:43.716 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2024-05-04T12:18:43.716 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2024-05-04T12:18:43.716 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2024-05-04T12:18:43.716 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2024-05-04T12:18:43.717 INFO:teuthology.orchestra.run.smithi111.stdout:/run/lock 2024-05-04T12:18:43.717 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2024-05-04T12:18:43.717 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2024-05-04T12:18:43.717 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2024-05-04T12:18:43.717 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:18:43.717 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2024-05-04T12:18:43.717 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2024-05-04T12:18:43.717 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2024-05-04T12:18:43.717 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2024-05-04T12:18:43.717 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/fs/nfsd 2024-05-04T12:18:43.717 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2024-05-04T12:18:43.717 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2024-05-04T12:18:43.717 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:18:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:/run/rpc_pipefs 2024-05-04T12:18:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2024-05-04T12:18:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2024-05-04T12:18:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-4 2024-05-04T12:18:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-5 2024-05-04T12:18:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-6 2024-05-04T12:18:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-7 2024-05-04T12:18:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2024-05-04T12:18:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T12:18:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T12:18:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-04T12:18:43.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:43.719 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-05-04T12:18:43.767 INFO:teuthology.orchestra.run.smithi111.stdout:45 2024-05-04T12:18:43.767 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-04T12:18:43.767 DEBUG:teuthology.orchestra.run.smithi111:> sudo stdin-killer -- python3 -c ' 2024-05-04T12:18:43.767 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2024-05-04T12:18:43.767 DEBUG:teuthology.orchestra.run.smithi111:> import re 2024-05-04T12:18:43.767 DEBUG:teuthology.orchestra.run.smithi111:> import os 2024-05-04T12:18:43.767 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2024-05-04T12:18:43.767 DEBUG:teuthology.orchestra.run.smithi111:> 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2024-05-04T12:18:43.768 DEBUG:teuthology.orchestra.run.smithi111:> return f 2024-05-04T12:18:43.769 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-04T12:18:43.769 DEBUG:teuthology.orchestra.run.smithi111:> 2024-05-04T12:18:43.769 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.1")) 2024-05-04T12:18:43.769 DEBUG:teuthology.orchestra.run.smithi111:> ' 2024-05-04T12:18:43.892 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.1.33492.asok 2024-05-04T12:18:43.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:43.904 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33492.asok 2024-05-04T12:18:43.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:43.904 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33492.asok status 2024-05-04T12:18:44.061 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-05-04T12:18:44.062 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2024-05-04T12:18:44.062 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "09dbd6bba52d19ba471c33bc15c008e6ad158ea6", 2024-05-04T12:18:44.062 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 19.0.0-2455-g09dbd6bb (09dbd6bba52d19ba471c33bc15c008e6ad158ea6) squid (dev)", 2024-05-04T12:18:44.062 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "1", 2024-05-04T12:18:44.062 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2024-05-04T12:18:44.062 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-04T12:18:44.062 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "33492", 2024-05-04T12:18:44.062 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2024-05-04T12:18:44.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-04T12:18:44.062 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2024-05-04T12:18:44.062 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2024-05-04T12:18:44.062 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4597, 2024-05-04T12:18:44.062 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2024-05-04T12:18:44.063 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2024-05-04T12:18:44.063 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2024-05-04T12:18:44.063 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 4597 2024-05-04T12:18:44.063 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-04T12:18:44.063 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2024-05-04T12:18:44.063 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-05-04T12:18:44.063 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2024-05-04T12:18:44.063 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2239007184 2024-05-04T12:18:44.063 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-04T12:18:44.063 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-04T12:18:44.063 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2024-05-04T12:18:44.063 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-05-04T12:18:44.063 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2024-05-04T12:18:44.064 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2239007184 2024-05-04T12:18:44.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-04T12:18:44.064 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.4597 192.168.0.1:0/2239007184", 2024-05-04T12:18:44.064 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.1:0/2239007184", 2024-05-04T12:18:44.064 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2024-05-04T12:18:44.064 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 7, 2024-05-04T12:18:44.064 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 21, 2024-05-04T12:18:44.064 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2024-05-04T12:18:44.064 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2024-05-04T12:18:44.064 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2024-05-04T12:18:44.064 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-05-04T12:18:44.075 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-04T12:18:44.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bb0329640 1 Processor -- start 2024-05-04T12:18:44.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bb0329640 1 -- start start 2024-05-04T12:18:44.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bb0329640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6ba806da80 0x7f6ba8067070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:44.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bb0329640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6ba806f6e0 0x7f6ba80cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:44.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bb0329640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ba80cfd90 0x7f6ba8161e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:44.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bb0329640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba80cf3c0 con 0x7f6ba806da80 2024-05-04T12:18:44.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bb0329640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba80ce7b0 con 0x7f6ba806f6e0 2024-05-04T12:18:44.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bb0329640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba80cea90 con 0x7f6ba80cfd90 2024-05-04T12:18:44.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bae0c5640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6ba806da80 0x7f6ba8067070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:44.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bae0c5640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6ba806da80 0x7f6ba8067070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36078/0 (socket says 172.21.15.52:36078) 2024-05-04T12:18:44.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bae0c5640 1 -- 172.21.15.52:0/3896971853 learned_addr learned my addr 172.21.15.52:0/3896971853 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:44.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bae8c6640 1 --2- 172.21.15.52:0/3896971853 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ba80cfd90 0x7f6ba8161e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:44.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bad8c4640 1 --2- 172.21.15.52:0/3896971853 >> v2:172.21.15.111:3300/0 conn(0x7f6ba806f6e0 0x7f6ba80cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:44.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bae0c5640 1 -- 172.21.15.52:0/3896971853 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ba80cfd90 msgr2=0x7f6ba8161e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:44.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bae0c5640 1 --2- 172.21.15.52:0/3896971853 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ba80cfd90 0x7f6ba8161e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:44.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bae0c5640 1 -- 172.21.15.52:0/3896971853 >> v2:172.21.15.111:3300/0 conn(0x7f6ba806f6e0 msgr2=0x7f6ba80cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:44.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bae0c5640 1 --2- 172.21.15.52:0/3896971853 >> v2:172.21.15.111:3300/0 conn(0x7f6ba806f6e0 0x7f6ba80cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:44.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bae0c5640 1 -- 172.21.15.52:0/3896971853 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba80ced70 con 0x7f6ba806da80 2024-05-04T12:18:44.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.205+0000 7f6bad8c4640 1 --2- 172.21.15.52:0/3896971853 >> v2:172.21.15.111:3300/0 conn(0x7f6ba806f6e0 0x7f6ba80cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:44.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.209+0000 7f6bae0c5640 1 --2- 172.21.15.52:0/3896971853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ba806da80 0x7f6ba8067070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80b1b30 tx=0x7f6ba401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa51e8c125dcba2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:44.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.209+0000 7f6bad0c3640 1 -- 172.21.15.52:0/3896971853 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40b1070 con 0x7f6ba806da80 2024-05-04T12:18:44.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.209+0000 7f6bad0c3640 1 -- 172.21.15.52:0/3896971853 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba40ac620 con 0x7f6ba806da80 2024-05-04T12:18:44.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.209+0000 7f6bad0c3640 1 -- 172.21.15.52:0/3896971853 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40ac9f0 con 0x7f6ba806da80 2024-05-04T12:18:44.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.209+0000 7f6bb0329640 1 -- 172.21.15.52:0/3896971853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ba806da80 msgr2=0x7f6ba8067070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:44.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.209+0000 7f6bb0329640 1 --2- 172.21.15.52:0/3896971853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ba806da80 0x7f6ba8067070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80b1b30 tx=0x7f6ba401c4f0 comp rx=0 tx=0).stop 2024-05-04T12:18:44.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.209+0000 7f6bb0329640 1 -- 172.21.15.52:0/3896971853 shutdown_connections 2024-05-04T12:18:44.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.209+0000 7f6bb0329640 1 --2- 172.21.15.52:0/3896971853 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ba80cfd90 0x7f6ba8161e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:44.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.209+0000 7f6bb0329640 1 --2- 172.21.15.52:0/3896971853 >> v2:172.21.15.111:3300/0 conn(0x7f6ba806f6e0 0x7f6ba80cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:44.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.209+0000 7f6bb0329640 1 --2- 172.21.15.52:0/3896971853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ba806da80 0x7f6ba8067070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:44.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.209+0000 7f6bb0329640 1 -- 172.21.15.52:0/3896971853 >> 172.21.15.52:0/3896971853 conn(0x7f6ba80b2fe0 msgr2=0x7f6ba80b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:44.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.209+0000 7f6bb0329640 1 -- 172.21.15.52:0/3896971853 shutdown_connections 2024-05-04T12:18:44.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.209+0000 7f6bb0329640 1 -- 172.21.15.52:0/3896971853 wait complete. 2024-05-04T12:18:44.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bb0329640 1 Processor -- start 2024-05-04T12:18:44.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bb0329640 1 -- start start 2024-05-04T12:18:44.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bb0329640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ba806da80 0x7f6ba8168630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:44.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bb0329640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba806f6e0 0x7f6ba8144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:44.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bb0329640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ba8144ed0 0x7f6ba8157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:44.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bb0329640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba815eb90 con 0x7f6ba806da80 2024-05-04T12:18:44.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bb0329640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba815ea10 con 0x7f6ba806f6e0 2024-05-04T12:18:44.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bb0329640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba815ed10 con 0x7f6ba8144ed0 2024-05-04T12:18:44.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bae0c5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ba806da80 0x7f6ba8168630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:44.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bae0c5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ba806da80 0x7f6ba8168630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36094/0 (socket says 172.21.15.52:36094) 2024-05-04T12:18:44.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bae0c5640 1 -- 172.21.15.52:0/2656353169 learned_addr learned my addr 172.21.15.52:0/2656353169 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:44.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bae8c6640 1 --2- 172.21.15.52:0/2656353169 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ba8144ed0 0x7f6ba8157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:44.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bad8c4640 1 --2- 172.21.15.52:0/2656353169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba806f6e0 0x7f6ba8144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:44.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bae0c5640 1 -- 172.21.15.52:0/2656353169 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ba8144ed0 msgr2=0x7f6ba8157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:44.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bae0c5640 1 --2- 172.21.15.52:0/2656353169 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ba8144ed0 0x7f6ba8157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:44.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bae0c5640 1 -- 172.21.15.52:0/2656353169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba806f6e0 msgr2=0x7f6ba8144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:44.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bae0c5640 1 --2- 172.21.15.52:0/2656353169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba806f6e0 0x7f6ba8144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:44.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bae0c5640 1 -- 172.21.15.52:0/2656353169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba81faf70 con 0x7f6ba806da80 2024-05-04T12:18:44.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bae0c5640 1 --2- 172.21.15.52:0/2656353169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ba806da80 0x7f6ba8168630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40af040 tx=0x7f6ba4019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:44.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6b9effd640 1 -- 172.21.15.52:0/2656353169 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40bb070 con 0x7f6ba806da80 2024-05-04T12:18:44.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6b9effd640 1 -- 172.21.15.52:0/2656353169 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba40b53f0 con 0x7f6ba806da80 2024-05-04T12:18:44.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6b9effd640 1 -- 172.21.15.52:0/2656353169 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40b1040 con 0x7f6ba806da80 2024-05-04T12:18:44.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bb0329640 1 -- 172.21.15.52:0/2656353169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba81fb250 con 0x7f6ba806da80 2024-05-04T12:18:44.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.213+0000 7f6bb0329640 1 -- 172.21.15.52:0/2656353169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba81fb810 con 0x7f6ba806da80 2024-05-04T12:18:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.217+0000 7f6bb0329640 1 -- 172.21.15.52:0/2656353169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba8062e50 con 0x7f6ba806da80 2024-05-04T12:18:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.217+0000 7f6b9effd640 1 -- 172.21.15.52:0/2656353169 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6ba40b57a0 con 0x7f6ba806da80 2024-05-04T12:18:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.217+0000 7f6b9effd640 1 --2- 172.21.15.52:0/2656353169 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6b8c0678c0 0x7f6b8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.217+0000 7f6b9effd640 1 -- 172.21.15.52:0/2656353169 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6ba411ee20 con 0x7f6ba806da80 2024-05-04T12:18:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.217+0000 7f6bad8c4640 1 --2- 172.21.15.52:0/2656353169 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6b8c0678c0 0x7f6b8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.217+0000 7f6bad8c4640 1 --2- 172.21.15.52:0/2656353169 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6b8c0678c0 0x7f6b8c069d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6ba81455b0 tx=0x7f6b98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:44.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.221+0000 7f6b9effd640 1 -- 172.21.15.52:0/2656353169 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6ba4016110 con 0x7f6ba806da80 2024-05-04T12:18:44.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.425+0000 7f6bb0329640 1 -- 172.21.15.52:0/2656353169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6ba815c170 con 0x7f6ba806da80 2024-05-04T12:18:44.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.425+0000 7f6b9effd640 1 -- 172.21.15.52:0/2656353169 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f6ba40ef310 con 0x7f6ba806da80 2024-05-04T12:18:44.429 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-04T12:18:44.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.429+0000 7f6bb0329640 1 -- 172.21.15.52:0/2656353169 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6b8c0678c0 msgr2=0x7f6b8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:44.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.429+0000 7f6bb0329640 1 --2- 172.21.15.52:0/2656353169 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6b8c0678c0 0x7f6b8c069d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6ba81455b0 tx=0x7f6b98008040 comp rx=0 tx=0).stop 2024-05-04T12:18:44.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.429+0000 7f6bb0329640 1 -- 172.21.15.52:0/2656353169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ba806da80 msgr2=0x7f6ba8168630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:44.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.429+0000 7f6bb0329640 1 --2- 172.21.15.52:0/2656353169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ba806da80 0x7f6ba8168630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40af040 tx=0x7f6ba4019290 comp rx=0 tx=0).stop 2024-05-04T12:18:44.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.429+0000 7f6bb0329640 1 -- 172.21.15.52:0/2656353169 shutdown_connections 2024-05-04T12:18:44.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.429+0000 7f6bb0329640 1 --2- 172.21.15.52:0/2656353169 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6b8c0678c0 0x7f6b8c069d80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:44.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.429+0000 7f6bb0329640 1 --2- 172.21.15.52:0/2656353169 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ba8144ed0 0x7f6ba8157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:44.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.429+0000 7f6bb0329640 1 --2- 172.21.15.52:0/2656353169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba806f6e0 0x7f6ba8144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:44.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.429+0000 7f6bb0329640 1 --2- 172.21.15.52:0/2656353169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ba806da80 0x7f6ba8168630 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:44.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.429+0000 7f6bb0329640 1 -- 172.21.15.52:0/2656353169 >> 172.21.15.52:0/2656353169 conn(0x7f6ba80b2fe0 msgr2=0x7f6ba815f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:44.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.429+0000 7f6bb0329640 1 -- 172.21.15.52:0/2656353169 shutdown_connections 2024-05-04T12:18:44.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:44.429+0000 7f6bb0329640 1 -- 172.21.15.52:0/2656353169 wait complete. 2024-05-04T12:18:44.448 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-04T12:18:44.448 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-04T12:18:44.448 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2024-05-04T12:18:44.449 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-04T12:18:44.449 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-04T12:18:44.449 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-04T12:18:44.449 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-04T12:18:44.449 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-04T12:18:44.449 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-04T12:18:44.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:44.449 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2024-05-04T12:18:44.456 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-04T12:18:44.456 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-04T12:18:44.456 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-05-04T12:18:44.456 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.456 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-05-04T12:18:44.456 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42507sec preferred_lft 42507sec 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-04T12:18:44.457 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 02:42:7f:88:9e:58 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:44.458 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-04T12:18:44.458 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.458 INFO:teuthology.orchestra.run.smithi111.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-04T12:18:44.458 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-04T12:18:44.458 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-04T12:18:44.458 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.458 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-04T12:18:44.458 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.458 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-04T12:18:44.458 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T12:18:44.458 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-04T12:18:44.458 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:44.459 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-04T12:18:44.509 INFO:teuthology.orchestra.run.smithi111.stdout:1 2024-05-04T12:18:44.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:44.510 DEBUG:teuthology.orchestra.run.smithi111:> route 2024-05-04T12:18:44.562 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2024-05-04T12:18:44.562 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-04T12:18:44.562 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-04T12:18:44.562 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-04T12:18:44.562 INFO:teuthology.orchestra.run.smithi111.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-04T12:18:44.562 INFO:teuthology.orchestra.run.smithi111.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-04T12:18:44.563 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-04T12:18:44.563 INFO:teuthology.orchestra.run.smithi111.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-04T12:18:44.563 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-04T12:18:44.563 INFO:teuthology.orchestra.run.smithi111.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-04T12:18:44.563 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-04T12:18:44.563 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:44.563 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-04T12:18:44.563 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-04T12:18:44.563 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-04T12:18:44.563 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-04T12:18:44.563 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-04T12:18:44.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:44.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:44.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:44.692 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-05-04T12:18:44.698 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-04T12:18:44.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:44.699 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2024-05-04T12:18:44.746 INFO:teuthology.orchestra.run.smithi111.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-04T12:18:44.746 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:44.746 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-04T12:18:44.746 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-04T12:18:44.747 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-04T12:18:44.747 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-04T12:18:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:44.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:44.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:44.863 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-04T12:18:44.875 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-04T12:18:44.875 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-04T12:18:44.875 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-05-04T12:18:44.875 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.876 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-05-04T12:18:44.876 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.876 INFO:teuthology.orchestra.run.smithi111.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-04T12:18:44.876 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 86:0a:cd:c0:bf:76 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-04T12:18:44.876 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-04T12:18:44.876 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.876 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::840a:cdff:fec0:bf76/64 scope link 2024-05-04T12:18:44.876 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:44.877 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-04T12:18:44.933 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-04T12:18:44.933 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-04T12:18:44.933 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-05-04T12:18:44.933 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.933 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-05-04T12:18:44.933 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.933 INFO:teuthology.orchestra.run.smithi111.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-04T12:18:44.933 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 86:0a:cd:c0:bf:76 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-04T12:18:44.933 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-04T12:18:44.933 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.934 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::840a:cdff:fec0:bf76/64 scope link 2024-05-04T12:18:44.934 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:18:44.934 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-04T12:18:44.935 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:44.935 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-04T12:18:44.935 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-04T12:18:44.935 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-04T12:18:44.935 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-04T12:18:44.935 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-04T12:18:44.935 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-04T12:18:44.935 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-04T12:18:45.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:45.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:45.080 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:18:45.080 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-04T12:18:45.080 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.1 up 2024-05-04T12:18:45.080 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.1 master ceph-brx 2024-05-04T12:18:45.080 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-04T12:18:45.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:18:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:45.146 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-05-04T12:18:45.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-04T12:18:45.147 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-04T12:18:45.198 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-04T12:18:45.198 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-04T12:18:45.198 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-04T12:18:45.249 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-05-04T12:18:45.308 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:18:45.355 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2024-05-04T12:18:45.355 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2024-05-04T12:18:45.355 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2024-05-04T12:18:45.355 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/run/lock 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/fs/nfsd 2024-05-04T12:18:45.356 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/run/rpc_pipefs 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-4 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-5 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-6 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-7 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-04T12:18:45.357 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-04T12:18:45.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:45.358 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-05-04T12:18:45.406 INFO:teuthology.orchestra.run.smithi111.stdout:45 2024-05-04T12:18:45.406 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-05-04T12:18:45.406 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-05-04T12:18:45.451 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-05-04T12:18:45.467 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:18:45.515 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2024-05-04T12:18:45.515 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2024-05-04T12:18:45.515 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2024-05-04T12:18:45.515 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2024-05-04T12:18:45.515 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2024-05-04T12:18:45.515 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2024-05-04T12:18:45.515 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2024-05-04T12:18:45.515 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/run/lock 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/fs/nfsd 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/run/rpc_pipefs 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2024-05-04T12:18:45.516 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2024-05-04T12:18:45.517 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-4 2024-05-04T12:18:45.517 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-5 2024-05-04T12:18:45.517 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-6 2024-05-04T12:18:45.517 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-7 2024-05-04T12:18:45.517 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2024-05-04T12:18:45.517 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T12:18:45.517 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T12:18:45.517 INFO:teuthology.orchestra.run.smithi111.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-04T12:18:45.517 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-04T12:18:45.517 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-04T12:18:45.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:45.517 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-05-04T12:18:45.535 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi111.stderr:2024-05-04T12:18:45.534+0000 7fe689797580 -1 init, newargv = 0x559a5c7bbe70 newargc=15 2024-05-04T12:18:45.535 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi111.stderr:ceph-fuse[33611]: starting ceph client 2024-05-04T12:18:45.539 INFO:teuthology.orchestra.run.smithi111.stdout:45 2024-05-04T12:18:45.541 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi111.stderr:ceph-fuse[33611]: starting fuse 2024-05-04T12:18:46.641 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-05-04T12:18:46.656 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:18:46.704 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2024-05-04T12:18:46.704 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2024-05-04T12:18:46.704 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2024-05-04T12:18:46.704 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2024-05-04T12:18:46.704 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2024-05-04T12:18:46.704 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2024-05-04T12:18:46.704 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2024-05-04T12:18:46.704 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2024-05-04T12:18:46.704 INFO:teuthology.orchestra.run.smithi111.stdout:/run/lock 2024-05-04T12:18:46.704 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2024-05-04T12:18:46.704 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2024-05-04T12:18:46.704 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/fs/nfsd 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/run/rpc_pipefs 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-4 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-5 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-6 2024-05-04T12:18:46.705 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-7 2024-05-04T12:18:46.706 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2024-05-04T12:18:46.706 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T12:18:46.706 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T12:18:46.706 INFO:teuthology.orchestra.run.smithi111.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-04T12:18:46.706 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-04T12:18:46.706 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-04T12:18:46.706 INFO:teuthology.orchestra.run.smithi111.stdout:/home/ubuntu/cephtest/mnt.3 2024-05-04T12:18:46.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:46.706 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-05-04T12:18:46.750 INFO:teuthology.orchestra.run.smithi111.stdout:45 2024-05-04T12:18:46.750 INFO:teuthology.orchestra.run.smithi111.stdout:46 2024-05-04T12:18:46.750 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-05-04T12:18:46.750 DEBUG:teuthology.orchestra.run.smithi111:> sudo stdin-killer -- python3 -c ' 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> import re 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> import os 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2024-05-04T12:18:46.751 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-04T12:18:46.752 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-04T12:18:46.752 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-04T12:18:46.752 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2024-05-04T12:18:46.752 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2024-05-04T12:18:46.752 DEBUG:teuthology.orchestra.run.smithi111:> return f 2024-05-04T12:18:46.752 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-04T12:18:46.752 DEBUG:teuthology.orchestra.run.smithi111:> 2024-05-04T12:18:46.752 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.3")) 2024-05-04T12:18:46.752 DEBUG:teuthology.orchestra.run.smithi111:> ' 2024-05-04T12:18:46.878 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.3.33611.asok 2024-05-04T12:18:46.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-04T12:18:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:18:46.890 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33611.asok 2024-05-04T12:18:46.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:46.890 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33611.asok status 2024-05-04T12:18:47.052 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-05-04T12:18:47.052 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2024-05-04T12:18:47.052 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "09dbd6bba52d19ba471c33bc15c008e6ad158ea6", 2024-05-04T12:18:47.052 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 19.0.0-2455-g09dbd6bb (09dbd6bba52d19ba471c33bc15c008e6ad158ea6) squid (dev)", 2024-05-04T12:18:47.052 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "3", 2024-05-04T12:18:47.052 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2024-05-04T12:18:47.052 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-05-04T12:18:47.052 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "33611", 2024-05-04T12:18:47.052 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2024-05-04T12:18:47.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-04T12:18:47.052 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2024-05-04T12:18:47.052 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2024-05-04T12:18:47.052 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4609, 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 4609 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.2:0", 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 4193460393 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.2:0", 2024-05-04T12:18:47.053 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 4193460393 2024-05-04T12:18:47.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-04T12:18:47.054 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.4609 192.168.0.2:0/4193460393", 2024-05-04T12:18:47.054 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.2:0/4193460393", 2024-05-04T12:18:47.054 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2024-05-04T12:18:47.054 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 7, 2024-05-04T12:18:47.054 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 21, 2024-05-04T12:18:47.054 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2024-05-04T12:18:47.054 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2024-05-04T12:18:47.054 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2024-05-04T12:18:47.054 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-05-04T12:18:47.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:47.066 DEBUG:teuthology.orchestra.run.smithi052:> stat --file-system '--printf=%T 2024-05-04T12:18:47.066 DEBUG:teuthology.orchestra.run.smithi052:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-04T12:18:47.073 INFO:teuthology.orchestra.run.smithi052.stdout:fuseblk 2024-05-04T12:18:47.074 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-04T12:18:47.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:47.074 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-04T12:18:47.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:47.131 DEBUG:teuthology.orchestra.run.smithi052:> stat --file-system '--printf=%T 2024-05-04T12:18:47.131 DEBUG:teuthology.orchestra.run.smithi052:> ' -- /home/ubuntu/cephtest/mnt.2 2024-05-04T12:18:47.181 INFO:teuthology.orchestra.run.smithi052.stdout:fuseblk 2024-05-04T12:18:47.181 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-05-04T12:18:47.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:47.181 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-04T12:18:47.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:47.240 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2024-05-04T12:18:47.240 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-04T12:18:47.247 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2024-05-04T12:18:47.247 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-04T12:18:47.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:47.248 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-04T12:18:47.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:47.303 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2024-05-04T12:18:47.303 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.3 2024-05-04T12:18:47.351 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2024-05-04T12:18:47.351 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-05-04T12:18:47.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:47.351 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-04T12:18:47.406 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-04T12:18:47.420 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:47.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a82905640 1 Processor -- start 2024-05-04T12:18:47.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a82905640 1 -- start start 2024-05-04T12:18:47.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a82905640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7a7c0aa0c0 0x7f7a7c0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:47.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a82905640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7a7c0aa550 0x7f7a7c0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:47.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a82905640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a7c06d1b0 con 0x7f7a7c15bf50 2024-05-04T12:18:47.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a82905640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f7a7c15ca40 con 0x7f7a7c0aa550 2024-05-04T12:18:47.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a82905640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f7a7c06d490 con 0x7f7a7c0aa0c0 2024-05-04T12:18:47.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a80ea2640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7a7c0aa550 0x7f7a7c0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:47.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a7bfff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f7a7c15bf50 0x7f7a7c15c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55040/0 (socket says 172.21.15.52:55040) 2024-05-04T12:18:47.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a7bfff640 1 -- 172.21.15.52:0/4257269590 learned_addr learned my addr 172.21.15.52:0/4257269590 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:47.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a80ea2640 1 -- 172.21.15.52:0/4257269590 >> v2:172.21.15.111:3300/0 conn(0x7f7a7c0aa0c0 msgr2=0x7f7a7c0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:47.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a7b7fe640 1 --2- 172.21.15.52:0/4257269590 >> v2:172.21.15.111:3300/0 conn(0x7f7a7c0aa0c0 0x7f7a7c0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:47.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a80ea2640 1 --2- 172.21.15.52:0/4257269590 >> v2:172.21.15.111:3300/0 conn(0x7f7a7c0aa0c0 0x7f7a7c0ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a80ea2640 1 -- 172.21.15.52:0/4257269590 >> v1:172.21.15.52:6789/0 conn(0x7f7a7c15bf50 legacy=0x7f7a7c15c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:47.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a80ea2640 1 -- 172.21.15.52:0/4257269590 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7c06d770 con 0x7f7a7c0aa550 2024-05-04T12:18:47.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.553+0000 7f7a7affd640 1 -- 172.21.15.52:0/4257269590 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2198999167 0 0) 0x7f7a7c06d1b0 con 0x7f7a7c15bf50 2024-05-04T12:18:47.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a80ea2640 1 --2- 172.21.15.52:0/4257269590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a7c0aa550 0x7f7a7c0cd090 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0b3ae0 tx=0x7f7a7401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c5fa35bc475e4d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:47.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a7affd640 1 -- 172.21.15.52:0/4257269590 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a740b2070 con 0x7f7a7c0aa550 2024-05-04T12:18:47.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a7affd640 1 -- 172.21.15.52:0/4257269590 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a740a59c0 con 0x7f7a7c0aa550 2024-05-04T12:18:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a7affd640 1 -- 172.21.15.52:0/4257269590 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a740a5d90 con 0x7f7a7c0aa550 2024-05-04T12:18:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 -- 172.21.15.52:0/4257269590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a7c0aa550 msgr2=0x7f7a7c0cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 --2- 172.21.15.52:0/4257269590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a7c0aa550 0x7f7a7c0cd090 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0b3ae0 tx=0x7f7a7401c900 comp rx=0 tx=0).stop 2024-05-04T12:18:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 -- 172.21.15.52:0/4257269590 shutdown_connections 2024-05-04T12:18:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 --2- 172.21.15.52:0/4257269590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a7c0aa550 0x7f7a7c0cd090 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 --2- 172.21.15.52:0/4257269590 >> v2:172.21.15.111:3300/0 conn(0x7f7a7c0aa0c0 0x7f7a7c0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 -- 172.21.15.52:0/4257269590 >> 172.21.15.52:0/4257269590 conn(0x7f7a7c0b14a0 msgr2=0x7f7a7c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 -- 172.21.15.52:0/4257269590 shutdown_connections 2024-05-04T12:18:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 -- 172.21.15.52:0/4257269590 wait complete. 2024-05-04T12:18:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 Processor -- start 2024-05-04T12:18:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 -- start start 2024-05-04T12:18:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a7c0aa0c0 0x7f7a7c150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a7c0aa550 0x7f7a7c14d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a7c15bf50 0x7f7a7c14a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c06ebe0 con 0x7f7a7c15bf50 2024-05-04T12:18:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c06ea60 con 0x7f7a7c0aa550 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a82905640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f7a7c06ed60 con 0x7f7a7c0aa0c0 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a80ea2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a7c15bf50 0x7f7a7c14a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a80ea2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a7c15bf50 0x7f7a7c14a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53502/0 (socket says 172.21.15.52:53502) 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.557+0000 7f7a80ea2640 1 -- 172.21.15.52:0/4104718420 learned_addr learned my addr 172.21.15.52:0/4104718420 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a7b7fe640 1 --2- 172.21.15.52:0/4104718420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a7c0aa550 0x7f7a7c14d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a7bfff640 1 --2- 172.21.15.52:0/4104718420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a7c0aa0c0 0x7f7a7c150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a80ea2640 1 -- 172.21.15.52:0/4104718420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a7c0aa0c0 msgr2=0x7f7a7c150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a80ea2640 1 --2- 172.21.15.52:0/4104718420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a7c0aa0c0 0x7f7a7c150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a80ea2640 1 -- 172.21.15.52:0/4104718420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a7c0aa550 msgr2=0x7f7a7c14d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a80ea2640 1 --2- 172.21.15.52:0/4104718420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a7c0aa550 0x7f7a7c14d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a80ea2640 1 -- 172.21.15.52:0/4104718420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7c151c70 con 0x7f7a7c15bf50 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a80ea2640 1 --2- 172.21.15.52:0/4104718420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a7c15bf50 0x7f7a7c14a810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7a740b0040 tx=0x7f7a740196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a78ff9640 1 -- 172.21.15.52:0/4104718420 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a740bd020 con 0x7f7a7c15bf50 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a78ff9640 1 -- 172.21.15.52:0/4104718420 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a740ad740 con 0x7f7a7c15bf50 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a82905640 1 -- 172.21.15.52:0/4104718420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7c14d670 con 0x7f7a7c15bf50 2024-05-04T12:18:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a78ff9640 1 -- 172.21.15.52:0/4104718420 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a740b2040 con 0x7f7a7c15bf50 2024-05-04T12:18:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a82905640 1 -- 172.21.15.52:0/4104718420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7c1516b0 con 0x7f7a7c15bf50 2024-05-04T12:18:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a78ff9640 1 -- 172.21.15.52:0/4104718420 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7a740c0020 con 0x7f7a7c15bf50 2024-05-04T12:18:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a82905640 1 -- 172.21.15.52:0/4104718420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a7c062e50 con 0x7f7a7c15bf50 2024-05-04T12:18:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a78ff9640 1 --2- 172.21.15.52:0/4104718420 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7a680678f0 0x7f7a68069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a78ff9640 1 -- 172.21.15.52:0/4104718420 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7a74120780 con 0x7f7a7c15bf50 2024-05-04T12:18:47.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a7bfff640 1 --2- 172.21.15.52:0/4104718420 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7a680678f0 0x7f7a68069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:47.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.561+0000 7f7a7bfff640 1 --2- 172.21.15.52:0/4104718420 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7a680678f0 0x7f7a68069db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c000f70 tx=0x7f7a6c007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:47.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.565+0000 7f7a78ff9640 1 -- 172.21.15.52:0/4104718420 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7a740c8020 con 0x7f7a7c15bf50 2024-05-04T12:18:47.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.729+0000 7f7a82905640 1 -- 172.21.15.52:0/4104718420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a7c075320 con 0x7f7a7c15bf50 2024-05-04T12:18:47.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.729+0000 7f7a78ff9640 1 -- 172.21.15.52:0/4104718420 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f7a74016520 con 0x7f7a7c15bf50 2024-05-04T12:18:47.733 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:47.733 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.733+0000 7f7a82905640 1 -- 172.21.15.52:0/4104718420 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7a680678f0 msgr2=0x7f7a68069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.733+0000 7f7a82905640 1 --2- 172.21.15.52:0/4104718420 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7a680678f0 0x7f7a68069db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c000f70 tx=0x7f7a6c007c10 comp rx=0 tx=0).stop 2024-05-04T12:18:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.733+0000 7f7a82905640 1 -- 172.21.15.52:0/4104718420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a7c15bf50 msgr2=0x7f7a7c14a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.733+0000 7f7a82905640 1 --2- 172.21.15.52:0/4104718420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a7c15bf50 0x7f7a7c14a810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7a740b0040 tx=0x7f7a740196a0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.733+0000 7f7a82905640 1 -- 172.21.15.52:0/4104718420 shutdown_connections 2024-05-04T12:18:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.733+0000 7f7a82905640 1 --2- 172.21.15.52:0/4104718420 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7a680678f0 0x7f7a68069db0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.733+0000 7f7a82905640 1 --2- 172.21.15.52:0/4104718420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a7c15bf50 0x7f7a7c14a810 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.733+0000 7f7a82905640 1 --2- 172.21.15.52:0/4104718420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a7c0aa550 0x7f7a7c14d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.733+0000 7f7a82905640 1 --2- 172.21.15.52:0/4104718420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a7c0aa0c0 0x7f7a7c150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.733+0000 7f7a82905640 1 -- 172.21.15.52:0/4104718420 >> 172.21.15.52:0/4104718420 conn(0x7f7a7c0b14a0 msgr2=0x7f7a7c0cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.733+0000 7f7a82905640 1 -- 172.21.15.52:0/4104718420 shutdown_connections 2024-05-04T12:18:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.733+0000 7f7a82905640 1 -- 172.21.15.52:0/4104718420 wait complete. 2024-05-04T12:18:47.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:18:47.751 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:18:47.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd12e160640 1 Processor -- start 2024-05-04T12:18:47.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd12e160640 1 -- start start 2024-05-04T12:18:47.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd12e160640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd12815bd90 0x7fd1280cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:47.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd12e160640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd12815c170 0x7fd1280cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:47.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd12e160640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd12806f370 0x7fd12806f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:47.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd12e160640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fd1280cc930 con 0x7fd12806f370 2024-05-04T12:18:47.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd12e160640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fd1280cd4b0 con 0x7fd12815bd90 2024-05-04T12:18:47.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd12e160640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fd12806fd60 con 0x7fd12815c170 2024-05-04T12:18:47.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd127fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd12806f370 0x7fd12806f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:47.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd127fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd12806f370 0x7fd12806f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53516/0 (socket says 172.21.15.52:53516) 2024-05-04T12:18:47.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd127fff640 1 -- 172.21.15.52:0/818156354 learned_addr learned my addr 172.21.15.52:0/818156354 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:47.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd126ffd640 1 --2- 172.21.15.52:0/818156354 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd12815c170 0x7fd1280cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:47.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd1277fe640 1 --2- 172.21.15.52:0/818156354 >> v2:172.21.15.111:3300/0 conn(0x7fd12815bd90 0x7fd1280cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:47.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd127fff640 1 -- 172.21.15.52:0/818156354 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd12815c170 msgr2=0x7fd1280cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:47.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd127fff640 1 --2- 172.21.15.52:0/818156354 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd12815c170 0x7fd1280cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd127fff640 1 -- 172.21.15.52:0/818156354 >> v2:172.21.15.111:3300/0 conn(0x7fd12815bd90 msgr2=0x7fd1280cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:47.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd127fff640 1 --2- 172.21.15.52:0/818156354 >> v2:172.21.15.111:3300/0 conn(0x7fd12815bd90 0x7fd1280cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd127fff640 1 -- 172.21.15.52:0/818156354 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1280ccb80 con 0x7fd12806f370 2024-05-04T12:18:47.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd1277fe640 1 --2- 172.21.15.52:0/818156354 >> v2:172.21.15.111:3300/0 conn(0x7fd12815bd90 0x7fd1280cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:47.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.877+0000 7fd126ffd640 1 --2- 172.21.15.52:0/818156354 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd12815c170 0x7fd1280cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:47.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd127fff640 1 --2- 172.21.15.52:0/818156354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd12806f370 0x7fd12806f820 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd1280cdcc0 tx=0x7fd12001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c856ff506f42f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:47.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd1267fc640 1 -- 172.21.15.52:0/818156354 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1200b0070 con 0x7fd12806f370 2024-05-04T12:18:47.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd1267fc640 1 -- 172.21.15.52:0/818156354 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1200ab600 con 0x7fd12806f370 2024-05-04T12:18:47.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd1267fc640 1 -- 172.21.15.52:0/818156354 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1200ab9d0 con 0x7fd12806f370 2024-05-04T12:18:47.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 -- 172.21.15.52:0/818156354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd12806f370 msgr2=0x7fd12806f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:47.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 --2- 172.21.15.52:0/818156354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd12806f370 0x7fd12806f820 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd1280cdcc0 tx=0x7fd12001c490 comp rx=0 tx=0).stop 2024-05-04T12:18:47.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 -- 172.21.15.52:0/818156354 shutdown_connections 2024-05-04T12:18:47.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 --2- 172.21.15.52:0/818156354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd12806f370 0x7fd12806f820 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 --2- 172.21.15.52:0/818156354 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd12815c170 0x7fd1280cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 --2- 172.21.15.52:0/818156354 >> v2:172.21.15.111:3300/0 conn(0x7fd12815bd90 0x7fd1280cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 -- 172.21.15.52:0/818156354 >> 172.21.15.52:0/818156354 conn(0x7fd128067730 msgr2=0x7fd1280b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:47.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 -- 172.21.15.52:0/818156354 shutdown_connections 2024-05-04T12:18:47.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 -- 172.21.15.52:0/818156354 wait complete. 2024-05-04T12:18:47.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 Processor -- start 2024-05-04T12:18:47.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 -- start start 2024-05-04T12:18:47.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd12806f370 0x7fd12814e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:47.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd12815bd90 0x7fd128148a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:47.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd12815c170 0x7fd1281546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:47.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd12815f460 con 0x7fd12806f370 2024-05-04T12:18:47.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd12815f2e0 con 0x7fd12815c170 2024-05-04T12:18:47.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fd12815f5e0 con 0x7fd12815bd90 2024-05-04T12:18:47.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd1277fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd12806f370 0x7fd12814e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:47.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd1277fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd12806f370 0x7fd12814e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53520/0 (socket says 172.21.15.52:53520) 2024-05-04T12:18:47.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd1277fe640 1 -- 172.21.15.52:0/3633664863 learned_addr learned my addr 172.21.15.52:0/3633664863 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:47.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd126ffd640 1 --2- 172.21.15.52:0/3633664863 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd12815bd90 0x7fd128148a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:47.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd127fff640 1 --2- 172.21.15.52:0/3633664863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd12815c170 0x7fd1281546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:47.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd1277fe640 1 -- 172.21.15.52:0/3633664863 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd12815bd90 msgr2=0x7fd128148a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:47.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd1277fe640 1 --2- 172.21.15.52:0/3633664863 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd12815bd90 0x7fd128148a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd1277fe640 1 -- 172.21.15.52:0/3633664863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd12815c170 msgr2=0x7fd1281546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:47.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd1277fe640 1 --2- 172.21.15.52:0/3633664863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd12815c170 0x7fd1281546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:47.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd1277fe640 1 -- 172.21.15.52:0/3633664863 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd128154fd0 con 0x7fd12806f370 2024-05-04T12:18:47.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd1277fe640 1 --2- 172.21.15.52:0/3633664863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd12806f370 0x7fd12814e7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd11800cf10 tx=0x7fd11800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:47.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd10ffff640 1 -- 172.21.15.52:0/3633664863 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11801a070 con 0x7fd12806f370 2024-05-04T12:18:47.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd10ffff640 1 -- 172.21.15.52:0/3633664863 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd11800d840 con 0x7fd12806f370 2024-05-04T12:18:47.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 -- 172.21.15.52:0/3633664863 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1281552b0 con 0x7fd12806f370 2024-05-04T12:18:47.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd10ffff640 1 -- 172.21.15.52:0/3633664863 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11800dc10 con 0x7fd12806f370 2024-05-04T12:18:47.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.881+0000 7fd12e160640 1 -- 172.21.15.52:0/3633664863 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd128155870 con 0x7fd12806f370 2024-05-04T12:18:47.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.885+0000 7fd12e160640 1 -- 172.21.15.52:0/3633664863 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd128062e50 con 0x7fd12806f370 2024-05-04T12:18:47.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.885+0000 7fd10ffff640 1 -- 172.21.15.52:0/3633664863 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd11801d070 con 0x7fd12806f370 2024-05-04T12:18:47.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.885+0000 7fd10ffff640 1 --2- 172.21.15.52:0/3633664863 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd1140678c0 0x7fd114069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:47.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.885+0000 7fd10ffff640 1 -- 172.21.15.52:0/3633664863 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd118086400 con 0x7fd12806f370 2024-05-04T12:18:47.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.885+0000 7fd126ffd640 1 --2- 172.21.15.52:0/3633664863 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd1140678c0 0x7fd114069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:47.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.885+0000 7fd126ffd640 1 --2- 172.21.15.52:0/3633664863 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd1140678c0 0x7fd114069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd11c006fd0 tx=0x7fd11c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:47.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:47.889+0000 7fd10ffff640 1 -- 172.21.15.52:0/3633664863 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd1180569a0 con 0x7fd12806f370 2024-05-04T12:18:48.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.053+0000 7fd12e160640 1 -- 172.21.15.52:0/3633664863 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd128070840 con 0x7fd12806f370 2024-05-04T12:18:48.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.053+0000 7fd10ffff640 1 -- 172.21.15.52:0/3633664863 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fd11805aca0 con 0x7fd12806f370 2024-05-04T12:18:48.056 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:18:48.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.053+0000 7fd12e160640 1 -- 172.21.15.52:0/3633664863 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd1140678c0 msgr2=0x7fd114069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.053+0000 7fd12e160640 1 --2- 172.21.15.52:0/3633664863 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd1140678c0 0x7fd114069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd11c006fd0 tx=0x7fd11c008040 comp rx=0 tx=0).stop 2024-05-04T12:18:48.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.053+0000 7fd12e160640 1 -- 172.21.15.52:0/3633664863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd12806f370 msgr2=0x7fd12814e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.053+0000 7fd12e160640 1 --2- 172.21.15.52:0/3633664863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd12806f370 0x7fd12814e7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd11800cf10 tx=0x7fd11800a710 comp rx=0 tx=0).stop 2024-05-04T12:18:48.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.053+0000 7fd12e160640 1 -- 172.21.15.52:0/3633664863 shutdown_connections 2024-05-04T12:18:48.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.053+0000 7fd12e160640 1 --2- 172.21.15.52:0/3633664863 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd1140678c0 0x7fd114069d80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.053+0000 7fd12e160640 1 --2- 172.21.15.52:0/3633664863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd12815c170 0x7fd1281546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.053+0000 7fd12e160640 1 --2- 172.21.15.52:0/3633664863 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd12815bd90 0x7fd128148a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.053+0000 7fd12e160640 1 --2- 172.21.15.52:0/3633664863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd12806f370 0x7fd12814e7a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.053+0000 7fd12e160640 1 -- 172.21.15.52:0/3633664863 >> 172.21.15.52:0/3633664863 conn(0x7fd128067730 msgr2=0x7fd1280b2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:48.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.053+0000 7fd12e160640 1 -- 172.21.15.52:0/3633664863 shutdown_connections 2024-05-04T12:18:48.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.053+0000 7fd12e160640 1 -- 172.21.15.52:0/3633664863 wait complete. 2024-05-04T12:18:48.075 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:18:48.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.197+0000 7f7594543640 1 Processor -- start 2024-05-04T12:18:48.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.197+0000 7f7594543640 1 -- start start 2024-05-04T12:18:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.197+0000 7f7594543640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f758c0cf770 0x7f758c06f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.197+0000 7f7594543640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f758c15c170 0x7f758c06f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.197+0000 7f7594543640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f758c0ccb80 con 0x7f758c15bd90 2024-05-04T12:18:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.197+0000 7f7594543640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f758c06fdf0 con 0x7f758c0cf770 2024-05-04T12:18:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.197+0000 7f7594543640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f758c0cc930 con 0x7f758c15c170 2024-05-04T12:18:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7591ade640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f758c0cf770 0x7f758c06f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7591ade640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f758c0cf770 0x7f758c06f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53530/0 (socket says 172.21.15.52:53530) 2024-05-04T12:18:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7591ade640 1 -- 172.21.15.52:0/3610812336 learned_addr learned my addr 172.21.15.52:0/3610812336 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7592ae0640 1 --2- 172.21.15.52:0/3610812336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f758c15c170 0x7f758c06f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7591ade640 1 -- 172.21.15.52:0/3610812336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f758c15c170 msgr2=0x7f758c06f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7591ade640 1 --2- 172.21.15.52:0/3610812336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f758c15c170 0x7f758c06f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7591ade640 1 -- 172.21.15.52:0/3610812336 >> v1:172.21.15.111:6789/0 conn(0x7f758c15bd90 legacy=0x7f758c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7591ade640 1 -- 172.21.15.52:0/3610812336 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c0cce60 con 0x7f758c0cf770 2024-05-04T12:18:48.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7592ae0640 1 --2- 172.21.15.52:0/3610812336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f758c15c170 0x7f758c06f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7591ade640 1 --2- 172.21.15.52:0/3610812336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f758c0cf770 0x7f758c06f370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f757c009870 tx=0x7f757c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b4492b278ed167 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f75912dd640 1 -- 172.21.15.52:0/3610812336 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c0b2070 con 0x7f758c0cf770 2024-05-04T12:18:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f75912dd640 1 -- 172.21.15.52:0/3610812336 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c0a9070 con 0x7f758c0cf770 2024-05-04T12:18:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f75912dd640 1 -- 172.21.15.52:0/3610812336 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c0ad880 con 0x7f758c0cf770 2024-05-04T12:18:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7594543640 1 -- 172.21.15.52:0/3610812336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f758c0cf770 msgr2=0x7f758c06f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7594543640 1 --2- 172.21.15.52:0/3610812336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f758c0cf770 0x7f758c06f370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f757c009870 tx=0x7f757c01c4d0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7594543640 1 -- 172.21.15.52:0/3610812336 shutdown_connections 2024-05-04T12:18:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7594543640 1 --2- 172.21.15.52:0/3610812336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f758c15c170 0x7f758c06f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7594543640 1 --2- 172.21.15.52:0/3610812336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f758c0cf770 0x7f758c06f370 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7594543640 1 -- 172.21.15.52:0/3610812336 >> 172.21.15.52:0/3610812336 conn(0x7f758c067730 msgr2=0x7f758c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7594543640 1 -- 172.21.15.52:0/3610812336 shutdown_connections 2024-05-04T12:18:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.201+0000 7f7594543640 1 -- 172.21.15.52:0/3610812336 wait complete. 2024-05-04T12:18:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7594543640 1 Processor -- start 2024-05-04T12:18:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7594543640 1 -- start start 2024-05-04T12:18:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7594543640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f758c0cf770 0x7f758c148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7594543640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f758c15bd90 0x7f758c15d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7594543640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f758c15c170 0x7f758c14d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7594543640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f758c06d820 con 0x7f758c0cf770 2024-05-04T12:18:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7594543640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f758c06d6a0 con 0x7f758c15bd90 2024-05-04T12:18:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7594543640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f758c06d9a0 con 0x7f758c15c170 2024-05-04T12:18:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f75922df640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f758c0cf770 0x7f758c148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f75922df640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f758c0cf770 0x7f758c148bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53532/0 (socket says 172.21.15.52:53532) 2024-05-04T12:18:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f75922df640 1 -- 172.21.15.52:0/563208959 learned_addr learned my addr 172.21.15.52:0/563208959 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7592ae0640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f758c15c170 0x7f758c14d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f75922df640 1 -- 172.21.15.52:0/563208959 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f758c15c170 msgr2=0x7f758c14d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f75922df640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f758c15c170 0x7f758c14d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7591ade640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f758c15bd90 0x7f758c15d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f75922df640 1 -- 172.21.15.52:0/563208959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f758c15bd90 msgr2=0x7f758c15d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f75922df640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f758c15bd90 0x7f758c15d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f75922df640 1 -- 172.21.15.52:0/563208959 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c15dcc0 con 0x7f758c0cf770 2024-05-04T12:18:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7592ae0640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f758c15c170 0x7f758c14d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f75922df640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f758c0cf770 0x7f758c148bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f758800c370 tx=0x7f758800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7582ffd640 1 -- 172.21.15.52:0/563208959 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75880052d0 con 0x7f758c0cf770 2024-05-04T12:18:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7582ffd640 1 -- 172.21.15.52:0/563208959 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7588005470 con 0x7f758c0cf770 2024-05-04T12:18:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7594543640 1 -- 172.21.15.52:0/563208959 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758c14a810 con 0x7f758c0cf770 2024-05-04T12:18:48.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7582ffd640 1 -- 172.21.15.52:0/563208959 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588007260 con 0x7f758c0cf770 2024-05-04T12:18:48.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7594543640 1 -- 172.21.15.52:0/563208959 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758c14add0 con 0x7f758c0cf770 2024-05-04T12:18:48.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.205+0000 7f7594543640 1 -- 172.21.15.52:0/563208959 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758c062e50 con 0x7f758c0cf770 2024-05-04T12:18:48.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.209+0000 7f7582ffd640 1 -- 172.21.15.52:0/563208959 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7588007400 con 0x7f758c0cf770 2024-05-04T12:18:48.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.209+0000 7f7582ffd640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f75700678c0 0x7f7570069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.209+0000 7f7582ffd640 1 -- 172.21.15.52:0/563208959 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7588085570 con 0x7f758c0cf770 2024-05-04T12:18:48.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.209+0000 7f7591ade640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f75700678c0 0x7f7570069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.209+0000 7f7591ade640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f75700678c0 0x7f7570069d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f757c0af000 tx=0x7f757c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:48.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.213+0000 7f7582ffd640 1 -- 172.21.15.52:0/563208959 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7588056190 con 0x7f758c0cf770 2024-05-04T12:18:48.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.373+0000 7f7594543640 1 -- 172.21.15.52:0/563208959 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f758c075320 con 0x7f758c0cf770 2024-05-04T12:18:48.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.373+0000 7f7582ffd640 1 -- 172.21.15.52:0/563208959 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f7588055a90 con 0x7f758c0cf770 2024-05-04T12:18:48.377 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:18:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.377+0000 7f7594543640 1 -- 172.21.15.52:0/563208959 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f75700678c0 msgr2=0x7f7570069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.377+0000 7f7594543640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f75700678c0 0x7f7570069d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f757c0af000 tx=0x7f757c008040 comp rx=0 tx=0).stop 2024-05-04T12:18:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.377+0000 7f7594543640 1 -- 172.21.15.52:0/563208959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f758c0cf770 msgr2=0x7f758c148bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.377+0000 7f7594543640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f758c0cf770 0x7f758c148bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f758800c370 tx=0x7f758800c830 comp rx=0 tx=0).stop 2024-05-04T12:18:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.377+0000 7f7594543640 1 -- 172.21.15.52:0/563208959 shutdown_connections 2024-05-04T12:18:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.377+0000 7f7594543640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f75700678c0 0x7f7570069d80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.377+0000 7f7594543640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f758c15c170 0x7f758c14d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.377+0000 7f7594543640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f758c15bd90 0x7f758c15d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.377+0000 7f7594543640 1 --2- 172.21.15.52:0/563208959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f758c0cf770 0x7f758c148bd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.377+0000 7f7594543640 1 -- 172.21.15.52:0/563208959 >> 172.21.15.52:0/563208959 conn(0x7f758c067730 msgr2=0x7f758c070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.377+0000 7f7594543640 1 -- 172.21.15.52:0/563208959 shutdown_connections 2024-05-04T12:18:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.377+0000 7f7594543640 1 -- 172.21.15.52:0/563208959 wait complete. 2024-05-04T12:18:48.396 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:18:48.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf03e80640 1 Processor -- start 2024-05-04T12:18:48.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf03e80640 1 -- start start 2024-05-04T12:18:48.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf03e80640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faefc145960 0x7faefc160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf03e80640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faefc06dbf0 con 0x7faefc0ccea0 2024-05-04T12:18:48.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf03e80640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faefc160ad0 con 0x7faefc15c590 2024-05-04T12:18:48.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf03e80640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7faefc06cfe0 con 0x7faefc145960 2024-05-04T12:18:48.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf0241d640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7faefc0ccea0 0x7faefc06e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55050/0 (socket says 172.21.15.52:55050) 2024-05-04T12:18:48.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf0241d640 1 -- 172.21.15.52:0/2398137982 learned_addr learned my addr 172.21.15.52:0/2398137982 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:48.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf01c1c640 1 --2- 172.21.15.52:0/2398137982 >> v2:172.21.15.111:3300/0 conn(0x7faefc145960 0x7faefc160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 -- 172.21.15.52:0/2398137982 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1317053772 0 0) 0x7faefc06dbf0 con 0x7faefc0ccea0 2024-05-04T12:18:48.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 -- 172.21.15.52:0/2398137982 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faee8003530 con 0x7faefc0ccea0 2024-05-04T12:18:48.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 -- 172.21.15.52:0/2398137982 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3179772371 0 0) 0x7faee8003530 con 0x7faefc0ccea0 2024-05-04T12:18:48.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 -- 172.21.15.52:0/2398137982 >> v2:172.21.15.111:3300/0 conn(0x7faefc145960 msgr2=0x7faefc160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:48.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 --2- 172.21.15.52:0/2398137982 >> v2:172.21.15.111:3300/0 conn(0x7faefc145960 0x7faefc160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 -- 172.21.15.52:0/2398137982 >> v1:172.21.15.111:6789/0 conn(0x7faefc15c590 legacy=0x7faefc0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 -- 172.21.15.52:0/2398137982 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc06d2c0 con 0x7faefc0ccea0 2024-05-04T12:18:48.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 -- 172.21.15.52:0/2398137982 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7faef4001a00 con 0x7faefc0ccea0 2024-05-04T12:18:48.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 -- 172.21.15.52:0/2398137982 >> v1:172.21.15.52:6789/0 conn(0x7faefc0ccea0 legacy=0x7faefc06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 --2- 172.21.15.52:0/2398137982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faee80039c0 0x7faee8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 --2- 172.21.15.52:0/2398137982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee8004810 0x7faee8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 --2- 172.21.15.52:0/2398137982 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faee8005220 0x7faee800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 -- 172.21.15.52:0/2398137982 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faefc06cfe0 con 0x7faee80039c0 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 -- 172.21.15.52:0/2398137982 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faee800c150 con 0x7faee8004810 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf00c1a640 1 -- 172.21.15.52:0/2398137982 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7faee800c350 con 0x7faee8005220 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf01c1c640 1 --2- 172.21.15.52:0/2398137982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faee80039c0 0x7faee8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf0141b640 1 --2- 172.21.15.52:0/2398137982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee8004810 0x7faee8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf01c1c640 1 -- 172.21.15.52:0/2398137982 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faee8005220 msgr2=0x7faee800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf01c1c640 1 --2- 172.21.15.52:0/2398137982 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faee8005220 0x7faee800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf01c1c640 1 -- 172.21.15.52:0/2398137982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee8004810 msgr2=0x7faee8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf01c1c640 1 --2- 172.21.15.52:0/2398137982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee8004810 0x7faee8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf01c1c640 1 -- 172.21.15.52:0/2398137982 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee8003530 con 0x7faee80039c0 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf0241d640 1 -- 172.21.15.52:0/2398137982 reap_dead start 2024-05-04T12:18:48.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.521+0000 7faf0141b640 1 --2- 172.21.15.52:0/2398137982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee8004810 0x7faee8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:48.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf01c1c640 1 --2- 172.21.15.52:0/2398137982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faee80039c0 0x7faee8003e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faef8009ae0 tx=0x7faef801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7decbb8e8a9fc65 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:48.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf00c1a640 1 -- 172.21.15.52:0/2398137982 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef80096b0 con 0x7faee80039c0 2024-05-04T12:18:48.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf00c1a640 1 -- 172.21.15.52:0/2398137982 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef80a6500 con 0x7faee80039c0 2024-05-04T12:18:48.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf00c1a640 1 -- 172.21.15.52:0/2398137982 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef80a68f0 con 0x7faee80039c0 2024-05-04T12:18:48.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 -- 172.21.15.52:0/2398137982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faee80039c0 msgr2=0x7faee8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 --2- 172.21.15.52:0/2398137982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faee80039c0 0x7faee8003e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faef8009ae0 tx=0x7faef801ca40 comp rx=0 tx=0).stop 2024-05-04T12:18:48.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 -- 172.21.15.52:0/2398137982 shutdown_connections 2024-05-04T12:18:48.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 --2- 172.21.15.52:0/2398137982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faee80039c0 0x7faee8003e00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 -- 172.21.15.52:0/2398137982 >> 172.21.15.52:0/2398137982 conn(0x7faefc067730 msgr2=0x7faefc0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:48.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 -- 172.21.15.52:0/2398137982 shutdown_connections 2024-05-04T12:18:48.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 -- 172.21.15.52:0/2398137982 wait complete. 2024-05-04T12:18:48.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 Processor -- start 2024-05-04T12:18:48.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 -- start start 2024-05-04T12:18:48.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee80039c0 0x7faefc1fb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faefc0711d0 0x7faefc1f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faefc137fd0 0x7faefc13c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faefc06cdc0 con 0x7faefc0711d0 2024-05-04T12:18:48.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faefc06cc40 con 0x7faee80039c0 2024-05-04T12:18:48.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf03e80640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7faefc0ccba0 con 0x7faefc137fd0 2024-05-04T12:18:48.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf0141b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faefc0711d0 0x7faefc1f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf0141b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faefc0711d0 0x7faefc1f86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53562/0 (socket says 172.21.15.52:53562) 2024-05-04T12:18:48.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.525+0000 7faf0141b640 1 -- 172.21.15.52:0/527418235 learned_addr learned my addr 172.21.15.52:0/527418235 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:48.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faf01c1c640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee80039c0 0x7faefc1fb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faf0241d640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faefc137fd0 0x7faefc13c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faf0141b640 1 -- 172.21.15.52:0/527418235 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faefc137fd0 msgr2=0x7faefc13c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faf0141b640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faefc137fd0 0x7faefc13c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faf0141b640 1 -- 172.21.15.52:0/527418235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee80039c0 msgr2=0x7faefc1fb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faf0141b640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee80039c0 0x7faefc1fb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faf0141b640 1 -- 172.21.15.52:0/527418235 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc071920 con 0x7faefc0711d0 2024-05-04T12:18:48.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faf0241d640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faefc137fd0 0x7faefc13c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:48.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faf0141b640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faefc0711d0 0x7faefc1f86b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faeec00d4a0 tx=0x7faeec00d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:48.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faef27fc640 1 -- 172.21.15.52:0/527418235 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec019070 con 0x7faefc0711d0 2024-05-04T12:18:48.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faef27fc640 1 -- 172.21.15.52:0/527418235 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec0054a0 con 0x7faefc0711d0 2024-05-04T12:18:48.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faf03e80640 1 -- 172.21.15.52:0/527418235 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faefc071650 con 0x7faefc0711d0 2024-05-04T12:18:48.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faef27fc640 1 -- 172.21.15.52:0/527418235 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec007270 con 0x7faefc0711d0 2024-05-04T12:18:48.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faf03e80640 1 -- 172.21.15.52:0/527418235 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faefc1fbba0 con 0x7faefc0711d0 2024-05-04T12:18:48.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faf03e80640 1 -- 172.21.15.52:0/527418235 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faefc062e50 con 0x7faefc0711d0 2024-05-04T12:18:48.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faef27fc640 1 -- 172.21.15.52:0/527418235 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faeec01c070 con 0x7faefc0711d0 2024-05-04T12:18:48.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faef27fc640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faee00678c0 0x7faee0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faf01c1c640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faee00678c0 0x7faee0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faef27fc640 1 -- 172.21.15.52:0/527418235 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faeec0857d0 con 0x7faefc0711d0 2024-05-04T12:18:48.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.529+0000 7faf01c1c640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faee00678c0 0x7faee0069d80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faef80af000 tx=0x7faef8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:48.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.533+0000 7faef27fc640 1 -- 172.21.15.52:0/527418235 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faeec023020 con 0x7faefc0711d0 2024-05-04T12:18:48.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.693+0000 7faf03e80640 1 -- 172.21.15.52:0/527418235 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faefc075320 con 0x7faefc0711d0 2024-05-04T12:18:48.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.693+0000 7faef27fc640 1 -- 172.21.15.52:0/527418235 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7faeec0563b0 con 0x7faefc0711d0 2024-05-04T12:18:48.698 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:18:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.697+0000 7faf03e80640 1 -- 172.21.15.52:0/527418235 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faee00678c0 msgr2=0x7faee0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.697+0000 7faf03e80640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faee00678c0 0x7faee0069d80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faef80af000 tx=0x7faef8008040 comp rx=0 tx=0).stop 2024-05-04T12:18:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.697+0000 7faf03e80640 1 -- 172.21.15.52:0/527418235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faefc0711d0 msgr2=0x7faefc1f86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.697+0000 7faf03e80640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faefc0711d0 0x7faefc1f86b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faeec00d4a0 tx=0x7faeec00d960 comp rx=0 tx=0).stop 2024-05-04T12:18:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.697+0000 7faf03e80640 1 -- 172.21.15.52:0/527418235 shutdown_connections 2024-05-04T12:18:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.697+0000 7faf03e80640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faee00678c0 0x7faee0069d80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.697+0000 7faf03e80640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faefc137fd0 0x7faefc13c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.697+0000 7faf03e80640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faefc0711d0 0x7faefc1f86b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.697+0000 7faf03e80640 1 --2- 172.21.15.52:0/527418235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faee80039c0 0x7faefc1fb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.697+0000 7faf03e80640 1 -- 172.21.15.52:0/527418235 >> 172.21.15.52:0/527418235 conn(0x7faefc067730 msgr2=0x7faefc15c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.697+0000 7faf03e80640 1 -- 172.21.15.52:0/527418235 shutdown_connections 2024-05-04T12:18:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.697+0000 7faf03e80640 1 -- 172.21.15.52:0/527418235 wait complete. 2024-05-04T12:18:48.717 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:48.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.825+0000 7f41cea13640 1 Processor -- start 2024-05-04T12:18:48.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.825+0000 7f41cea13640 1 -- start start 2024-05-04T12:18:48.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41cea13640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41c80aa0c0 0x7f41c80aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41cea13640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f41c80ccb50 0x7f41c80cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41cea13640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41c806d1b0 con 0x7f41c815bf50 2024-05-04T12:18:48.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41cea13640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f41c815ca40 con 0x7f41c80ccb50 2024-05-04T12:18:48.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41cea13640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f41c806d490 con 0x7f41c80aa0c0 2024-05-04T12:18:48.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c7fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f41c815bf50 0x7f41c815c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55062/0 (socket says 172.21.15.52:55062) 2024-05-04T12:18:48.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c7fff640 1 -- 172.21.15.52:0/4027366334 learned_addr learned my addr 172.21.15.52:0/4027366334 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:48.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c77fe640 1 --2- 172.21.15.52:0/4027366334 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41c80aa0c0 0x7f41c80aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41ccfb0640 1 --2- 172.21.15.52:0/4027366334 >> v2:172.21.15.111:3300/0 conn(0x7f41c80ccb50 0x7f41c80cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 -- 172.21.15.52:0/4027366334 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3446365993 0 0) 0x7f41c806d1b0 con 0x7f41c815bf50 2024-05-04T12:18:48.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 -- 172.21.15.52:0/4027366334 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41b4003530 con 0x7f41c815bf50 2024-05-04T12:18:48.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 -- 172.21.15.52:0/4027366334 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2328700540 0 0) 0x7f41b4003530 con 0x7f41c815bf50 2024-05-04T12:18:48.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 -- 172.21.15.52:0/4027366334 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41c80aa0c0 msgr2=0x7f41c80aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:48.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 --2- 172.21.15.52:0/4027366334 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41c80aa0c0 0x7f41c80aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 -- 172.21.15.52:0/4027366334 >> v2:172.21.15.111:3300/0 conn(0x7f41c80ccb50 msgr2=0x7f41c80cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:48.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 --2- 172.21.15.52:0/4027366334 >> v2:172.21.15.111:3300/0 conn(0x7f41c80ccb50 0x7f41c80cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 -- 172.21.15.52:0/4027366334 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c806d770 con 0x7f41c815bf50 2024-05-04T12:18:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41ccfb0640 1 --2- 172.21.15.52:0/4027366334 >> v2:172.21.15.111:3300/0 conn(0x7f41c80ccb50 0x7f41c80cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 -- 172.21.15.52:0/4027366334 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f41b8001a00 con 0x7f41c815bf50 2024-05-04T12:18:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 -- 172.21.15.52:0/4027366334 >> v1:172.21.15.52:6789/0 conn(0x7f41c815bf50 legacy=0x7f41c815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 --2- 172.21.15.52:0/4027366334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41b40039c0 0x7f41b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 --2- 172.21.15.52:0/4027366334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41b4004810 0x7f41b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 --2- 172.21.15.52:0/4027366334 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41c80aa0c0 0x7f41b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 -- 172.21.15.52:0/4027366334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f41c815ca40 con 0x7f41b40039c0 2024-05-04T12:18:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 -- 172.21.15.52:0/4027366334 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f41c806d490 con 0x7f41b4004810 2024-05-04T12:18:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 -- 172.21.15.52:0/4027366334 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f41b400b9d0 con 0x7f41c80aa0c0 2024-05-04T12:18:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c7fff640 1 --2- 172.21.15.52:0/4027366334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41b40039c0 0x7f41b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c77fe640 1 --2- 172.21.15.52:0/4027366334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41b4004810 0x7f41b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c7fff640 1 -- 172.21.15.52:0/4027366334 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41c80aa0c0 msgr2=0x7f41b40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c7fff640 1 --2- 172.21.15.52:0/4027366334 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41c80aa0c0 0x7f41b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c7fff640 1 -- 172.21.15.52:0/4027366334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41b4004810 msgr2=0x7f41b4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:48.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c7fff640 1 --2- 172.21.15.52:0/4027366334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41b4004810 0x7f41b4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c7fff640 1 -- 172.21.15.52:0/4027366334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b4003530 con 0x7f41b40039c0 2024-05-04T12:18:48.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c7fff640 1 --2- 172.21.15.52:0/4027366334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41b40039c0 0x7f41b4003de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f41b800a920 tx=0x7f41b801d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1629fc6b644e51f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:48.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 -- 172.21.15.52:0/4027366334 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b800a4c0 con 0x7f41b40039c0 2024-05-04T12:18:48.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 -- 172.21.15.52:0/4027366334 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b80ad630 con 0x7f41b40039c0 2024-05-04T12:18:48.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.829+0000 7f41c6ffd640 1 -- 172.21.15.52:0/4027366334 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b80ada60 con 0x7f41b40039c0 2024-05-04T12:18:48.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 -- 172.21.15.52:0/4027366334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41b40039c0 msgr2=0x7f41b4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 --2- 172.21.15.52:0/4027366334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41b40039c0 0x7f41b4003de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f41b800a920 tx=0x7f41b801d440 comp rx=0 tx=0).stop 2024-05-04T12:18:48.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41ccfb0640 1 -- 172.21.15.52:0/4027366334 reap_dead start 2024-05-04T12:18:48.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 -- 172.21.15.52:0/4027366334 shutdown_connections 2024-05-04T12:18:48.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 -- 172.21.15.52:0/4027366334 >> 172.21.15.52:0/4027366334 conn(0x7f41c80b14a0 msgr2=0x7f41c80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:48.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 -- 172.21.15.52:0/4027366334 shutdown_connections 2024-05-04T12:18:48.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 -- 172.21.15.52:0/4027366334 wait complete. 2024-05-04T12:18:48.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 Processor -- start 2024-05-04T12:18:48.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 -- start start 2024-05-04T12:18:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41c8156af0 0x7f41c814c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41c814cad0 0x7f41c813c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41c814c0e0 0x7f41c813cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f41c806ea60 con 0x7f41c814c0e0 2024-05-04T12:18:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f41c806cf70 con 0x7f41c8156af0 2024-05-04T12:18:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f41c806ebe0 con 0x7f41c814cad0 2024-05-04T12:18:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41ccfb0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41c814c0e0 0x7f41c813cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41ccfb0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41c814c0e0 0x7f41c813cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53584/0 (socket says 172.21.15.52:53584) 2024-05-04T12:18:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41ccfb0640 1 -- 172.21.15.52:0/2860954886 learned_addr learned my addr 172.21.15.52:0/2860954886 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41c77fe640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41c814cad0 0x7f41c813c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41c7fff640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41c8156af0 0x7f41c814c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41ccfb0640 1 -- 172.21.15.52:0/2860954886 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41c814cad0 msgr2=0x7f41c813c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41ccfb0640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41c814cad0 0x7f41c813c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41ccfb0640 1 -- 172.21.15.52:0/2860954886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41c8156af0 msgr2=0x7f41c814c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41ccfb0640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41c8156af0 0x7f41c814c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41ccfb0640 1 -- 172.21.15.52:0/2860954886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c814cf50 con 0x7f41c814c0e0 2024-05-04T12:18:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41c7fff640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41c8156af0 0x7f41c814c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41ccfb0640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41c814c0e0 0x7f41c813cb30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f41c80aac10 tx=0x7f41c000e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41c4ff9640 1 -- 172.21.15.52:0/2860954886 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c001a070 con 0x7f41c814c0e0 2024-05-04T12:18:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41c4ff9640 1 -- 172.21.15.52:0/2860954886 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c0004250 con 0x7f41c814c0e0 2024-05-04T12:18:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 -- 172.21.15.52:0/2860954886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c814b320 con 0x7f41c814c0e0 2024-05-04T12:18:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41c4ff9640 1 -- 172.21.15.52:0/2860954886 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c000f040 con 0x7f41c814c0e0 2024-05-04T12:18:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 -- 172.21.15.52:0/2860954886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c814b8e0 con 0x7f41c814c0e0 2024-05-04T12:18:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.833+0000 7f41cea13640 1 -- 172.21.15.52:0/2860954886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c8062e50 con 0x7f41c814c0e0 2024-05-04T12:18:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.837+0000 7f41c4ff9640 1 -- 172.21.15.52:0/2860954886 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f41c001d070 con 0x7f41c814c0e0 2024-05-04T12:18:48.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.837+0000 7f41c4ff9640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f41a80678f0 0x7f41a8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:48.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.837+0000 7f41c7fff640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f41a80678f0 0x7f41a8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:48.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.837+0000 7f41c4ff9640 1 -- 172.21.15.52:0/2860954886 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f41c00849f0 con 0x7f41c814c0e0 2024-05-04T12:18:48.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.837+0000 7f41c7fff640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f41a80678f0 0x7f41a8069db0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f41b80ad000 tx=0x7f41b8007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:48.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.841+0000 7f41c4ff9640 1 -- 172.21.15.52:0/2860954886 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f41c004f5c0 con 0x7f41c814c0e0 2024-05-04T12:18:49.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.997+0000 7f41cea13640 1 -- 172.21.15.52:0/2860954886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41c8075320 con 0x7f41c814c0e0 2024-05-04T12:18:49.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:48.997+0000 7f41c4ff9640 1 -- 172.21.15.52:0/2860954886 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f41c0055400 con 0x7f41c814c0e0 2024-05-04T12:18:49.002 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:49.003 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:49.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.001+0000 7f41cea13640 1 -- 172.21.15.52:0/2860954886 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f41a80678f0 msgr2=0x7f41a8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.001+0000 7f41cea13640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f41a80678f0 0x7f41a8069db0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f41b80ad000 tx=0x7f41b8007040 comp rx=0 tx=0).stop 2024-05-04T12:18:49.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.001+0000 7f41cea13640 1 -- 172.21.15.52:0/2860954886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41c814c0e0 msgr2=0x7f41c813cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.001+0000 7f41cea13640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41c814c0e0 0x7f41c813cb30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f41c80aac10 tx=0x7f41c000e770 comp rx=0 tx=0).stop 2024-05-04T12:18:49.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.001+0000 7f41cea13640 1 -- 172.21.15.52:0/2860954886 shutdown_connections 2024-05-04T12:18:49.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.001+0000 7f41cea13640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f41a80678f0 0x7f41a8069db0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.001+0000 7f41cea13640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41c814c0e0 0x7f41c813cb30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.001+0000 7f41cea13640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41c814cad0 0x7f41c813c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.001+0000 7f41cea13640 1 --2- 172.21.15.52:0/2860954886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41c8156af0 0x7f41c814c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.001+0000 7f41cea13640 1 -- 172.21.15.52:0/2860954886 >> 172.21.15.52:0/2860954886 conn(0x7f41c80b14a0 msgr2=0x7f41c81591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:49.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.001+0000 7f41cea13640 1 -- 172.21.15.52:0/2860954886 shutdown_connections 2024-05-04T12:18:49.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.001+0000 7f41cea13640 1 -- 172.21.15.52:0/2860954886 wait complete. 2024-05-04T12:18:49.022 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:18:49.022 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:18:49.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b48142640 1 Processor -- start 2024-05-04T12:18:49.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b48142640 1 -- start start 2024-05-04T12:18:49.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b48142640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b4015c170 0x7f5b40165e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b48142640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b400ccb80 con 0x7f5b400cf770 2024-05-04T12:18:49.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b48142640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b400cce60 con 0x7f5b4015bd90 2024-05-04T12:18:49.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b48142640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5b400cc930 con 0x7f5b4015c170 2024-05-04T12:18:49.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b456dd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f5b400cf770 0x7f5b4006f370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55078/0 (socket says 172.21.15.52:55078) 2024-05-04T12:18:49.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b456dd640 1 -- 172.21.15.52:0/666586938 learned_addr learned my addr 172.21.15.52:0/666586938 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:49.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b466df640 1 --2- 172.21.15.52:0/666586938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b4015c170 0x7f5b40165e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 -- 172.21.15.52:0/666586938 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1134070677 0 0) 0x7f5b400ccb80 con 0x7f5b400cf770 2024-05-04T12:18:49.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 -- 172.21.15.52:0/666586938 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b2c003530 con 0x7f5b400cf770 2024-05-04T12:18:49.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 -- 172.21.15.52:0/666586938 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1353096323 0 0) 0x7f5b2c003530 con 0x7f5b400cf770 2024-05-04T12:18:49.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 -- 172.21.15.52:0/666586938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b4015c170 msgr2=0x7f5b40165e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:49.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 --2- 172.21.15.52:0/666586938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b4015c170 0x7f5b40165e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 -- 172.21.15.52:0/666586938 >> v1:172.21.15.111:6789/0 conn(0x7f5b4015bd90 legacy=0x7f5b400cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 -- 172.21.15.52:0/666586938 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b400cd140 con 0x7f5b400cf770 2024-05-04T12:18:49.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 -- 172.21.15.52:0/666586938 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f5b30001680 con 0x7f5b400cf770 2024-05-04T12:18:49.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 -- 172.21.15.52:0/666586938 >> v1:172.21.15.52:6789/0 conn(0x7f5b400cf770 legacy=0x7f5b4006f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 --2- 172.21.15.52:0/666586938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2c0039f0 0x7f5b2c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 --2- 172.21.15.52:0/666586938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b2c004810 0x7f5b2c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 --2- 172.21.15.52:0/666586938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b4015c170 0x7f5b2c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 -- 172.21.15.52:0/666586938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b400cc930 con 0x7f5b2c0039f0 2024-05-04T12:18:49.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 -- 172.21.15.52:0/666586938 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c00bb30 con 0x7f5b2c004810 2024-05-04T12:18:49.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b44edc640 1 -- 172.21.15.52:0/666586938 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5b2c00bd30 con 0x7f5b4015c170 2024-05-04T12:18:49.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b45ede640 1 --2- 172.21.15.52:0/666586938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2c0039f0 0x7f5b2c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b466df640 1 --2- 172.21.15.52:0/666586938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b4015c170 0x7f5b2c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b456dd640 1 --2- 172.21.15.52:0/666586938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b2c004810 0x7f5b2c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b45ede640 1 -- 172.21.15.52:0/666586938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b4015c170 msgr2=0x7f5b2c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:49.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b45ede640 1 --2- 172.21.15.52:0/666586938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b4015c170 0x7f5b2c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b45ede640 1 -- 172.21.15.52:0/666586938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b2c004810 msgr2=0x7f5b2c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:49.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b45ede640 1 --2- 172.21.15.52:0/666586938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b2c004810 0x7f5b2c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.149+0000 7f5b45ede640 1 -- 172.21.15.52:0/666586938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c003530 con 0x7f5b2c0039f0 2024-05-04T12:18:49.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.153+0000 7f5b45ede640 1 --2- 172.21.15.52:0/666586938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2c0039f0 0x7f5b2c003e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c001780 tx=0x7f5b3c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5844ebb7488c6aaf server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:49.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.153+0000 7f5b44edc640 1 -- 172.21.15.52:0/666586938 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c0013a0 con 0x7f5b2c0039f0 2024-05-04T12:18:49.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.153+0000 7f5b44edc640 1 -- 172.21.15.52:0/666586938 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b3c0052f0 con 0x7f5b2c0039f0 2024-05-04T12:18:49.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.153+0000 7f5b44edc640 1 -- 172.21.15.52:0/666586938 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c0a4e60 con 0x7f5b2c0039f0 2024-05-04T12:18:49.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.153+0000 7f5b48142640 1 -- 172.21.15.52:0/666586938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2c0039f0 msgr2=0x7f5b2c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.153+0000 7f5b48142640 1 --2- 172.21.15.52:0/666586938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2c0039f0 0x7f5b2c003e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c001780 tx=0x7f5b3c01c420 comp rx=0 tx=0).stop 2024-05-04T12:18:49.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.153+0000 7f5b466df640 1 -- 172.21.15.52:0/666586938 reap_dead start 2024-05-04T12:18:49.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.153+0000 7f5b48142640 1 -- 172.21.15.52:0/666586938 shutdown_connections 2024-05-04T12:18:49.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.153+0000 7f5b48142640 1 -- 172.21.15.52:0/666586938 >> 172.21.15.52:0/666586938 conn(0x7f5b40067730 msgr2=0x7f5b400b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:49.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.153+0000 7f5b48142640 1 -- 172.21.15.52:0/666586938 shutdown_connections 2024-05-04T12:18:49.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.153+0000 7f5b48142640 1 -- 172.21.15.52:0/666586938 wait complete. 2024-05-04T12:18:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.153+0000 7f5b48142640 1 Processor -- start 2024-05-04T12:18:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.153+0000 7f5b48142640 1 -- start start 2024-05-04T12:18:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b48142640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2c0039f0 0x7f5b40149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b48142640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b400b0910 0x7f5b4014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b48142640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b400b0d90 0x7f5b40150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b48142640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40166a00 con 0x7f5b2c0039f0 2024-05-04T12:18:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b48142640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40166880 con 0x7f5b400b0d90 2024-05-04T12:18:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b48142640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5b40166b80 con 0x7f5b400b0910 2024-05-04T12:18:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b45ede640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2c0039f0 0x7f5b40149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b45ede640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2c0039f0 0x7f5b40149720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53602/0 (socket says 172.21.15.52:53602) 2024-05-04T12:18:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b45ede640 1 -- 172.21.15.52:0/505993220 learned_addr learned my addr 172.21.15.52:0/505993220 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b466df640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b400b0d90 0x7f5b40150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b456dd640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b400b0910 0x7f5b4014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b45ede640 1 -- 172.21.15.52:0/505993220 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b400b0910 msgr2=0x7f5b4014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b45ede640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b400b0910 0x7f5b4014fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b45ede640 1 -- 172.21.15.52:0/505993220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b400b0d90 msgr2=0x7f5b40150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b45ede640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b400b0d90 0x7f5b40150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b45ede640 1 -- 172.21.15.52:0/505993220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b40155170 con 0x7f5b2c0039f0 2024-05-04T12:18:49.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b466df640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b400b0d90 0x7f5b40150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:49.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b456dd640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b400b0910 0x7f5b4014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:49.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b45ede640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2c0039f0 0x7f5b40149720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c001780 tx=0x7f5b2c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:49.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b367fc640 1 -- 172.21.15.52:0/505993220 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c007a40 con 0x7f5b2c0039f0 2024-05-04T12:18:49.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b367fc640 1 -- 172.21.15.52:0/505993220 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b3c0bb070 con 0x7f5b2c0039f0 2024-05-04T12:18:49.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b48142640 1 -- 172.21.15.52:0/505993220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b401f5480 con 0x7f5b2c0039f0 2024-05-04T12:18:49.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b367fc640 1 -- 172.21.15.52:0/505993220 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c000ee0 con 0x7f5b2c0039f0 2024-05-04T12:18:49.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b48142640 1 -- 172.21.15.52:0/505993220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b401f8f50 con 0x7f5b2c0039f0 2024-05-04T12:18:49.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b48142640 1 -- 172.21.15.52:0/505993220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b40062e50 con 0x7f5b2c0039f0 2024-05-04T12:18:49.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b367fc640 1 -- 172.21.15.52:0/505993220 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5b3c0b4040 con 0x7f5b2c0039f0 2024-05-04T12:18:49.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b367fc640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5b24067a90 0x7f5b24069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b367fc640 1 -- 172.21.15.52:0/505993220 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5b3c11fa60 con 0x7f5b2c0039f0 2024-05-04T12:18:49.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b456dd640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5b24067a90 0x7f5b24069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.157+0000 7f5b456dd640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5b24067a90 0x7f5b24069f50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5b400ae920 tx=0x7f5b30000ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:49.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.161+0000 7f5b367fc640 1 -- 172.21.15.52:0/505993220 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5b3c016130 con 0x7f5b2c0039f0 2024-05-04T12:18:49.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.329+0000 7f5b48142640 1 -- 172.21.15.52:0/505993220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5b401553d0 con 0x7f5b2c0039f0 2024-05-04T12:18:49.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.333+0000 7f5b367fc640 1 -- 172.21.15.52:0/505993220 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f5b3c0f05a0 con 0x7f5b2c0039f0 2024-05-04T12:18:49.335 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:18:49.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.333+0000 7f5b48142640 1 -- 172.21.15.52:0/505993220 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5b24067a90 msgr2=0x7f5b24069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.333+0000 7f5b48142640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5b24067a90 0x7f5b24069f50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5b400ae920 tx=0x7f5b30000ee0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.333+0000 7f5b48142640 1 -- 172.21.15.52:0/505993220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2c0039f0 msgr2=0x7f5b40149720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.333+0000 7f5b48142640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2c0039f0 0x7f5b40149720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c001780 tx=0x7f5b2c005840 comp rx=0 tx=0).stop 2024-05-04T12:18:49.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.333+0000 7f5b48142640 1 -- 172.21.15.52:0/505993220 shutdown_connections 2024-05-04T12:18:49.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.333+0000 7f5b48142640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5b24067a90 0x7f5b24069f50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.333+0000 7f5b48142640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b400b0d90 0x7f5b40150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.333+0000 7f5b48142640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b400b0910 0x7f5b4014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.333+0000 7f5b48142640 1 --2- 172.21.15.52:0/505993220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2c0039f0 0x7f5b40149720 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.333+0000 7f5b48142640 1 -- 172.21.15.52:0/505993220 >> 172.21.15.52:0/505993220 conn(0x7f5b40067730 msgr2=0x7f5b400b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:49.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.333+0000 7f5b48142640 1 -- 172.21.15.52:0/505993220 shutdown_connections 2024-05-04T12:18:49.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.333+0000 7f5b48142640 1 -- 172.21.15.52:0/505993220 wait complete. 2024-05-04T12:18:49.354 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:18:49.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bf0165640 1 Processor -- start 2024-05-04T12:18:49.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bf0165640 1 -- start start 2024-05-04T12:18:49.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bf0165640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8be8145960 0x7f8be8160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bf0165640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8be80ccea0 0x7f8be806e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bf0165640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be806dbf0 con 0x7f8be815c590 2024-05-04T12:18:49.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bf0165640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f8be8160ad0 con 0x7f8be80ccea0 2024-05-04T12:18:49.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bf0165640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f8be806cfe0 con 0x7f8be8145960 2024-05-04T12:18:49.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bee702640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8be80ccea0 0x7f8be806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bed700640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f8be815c590 0x7f8be80cc790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55094/0 (socket says 172.21.15.52:55094) 2024-05-04T12:18:49.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bed700640 1 -- 172.21.15.52:0/1406702608 learned_addr learned my addr 172.21.15.52:0/1406702608 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:49.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bedf01640 1 --2- 172.21.15.52:0/1406702608 >> v2:172.21.15.111:3300/0 conn(0x7f8be8145960 0x7f8be8160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bee702640 1 -- 172.21.15.52:0/1406702608 >> v2:172.21.15.111:3300/0 conn(0x7f8be8145960 msgr2=0x7f8be8160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:49.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bee702640 1 --2- 172.21.15.52:0/1406702608 >> v2:172.21.15.111:3300/0 conn(0x7f8be8145960 0x7f8be8160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bee702640 1 -- 172.21.15.52:0/1406702608 >> v1:172.21.15.52:6789/0 conn(0x7f8be815c590 legacy=0x7f8be80cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.485+0000 7f8bee702640 1 -- 172.21.15.52:0/1406702608 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be806d2c0 con 0x7f8be80ccea0 2024-05-04T12:18:49.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bee702640 1 --2- 172.21.15.52:0/1406702608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be80ccea0 0x7f8be806e080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8be8073670 tx=0x7f8be001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0c566175c8b262 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:49.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8beceff640 1 -- 172.21.15.52:0/1406702608 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be00b2070 con 0x7f8be80ccea0 2024-05-04T12:18:49.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8beceff640 1 -- 172.21.15.52:0/1406702608 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be00a5930 con 0x7f8be80ccea0 2024-05-04T12:18:49.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8beceff640 1 -- 172.21.15.52:0/1406702608 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be00a5d00 con 0x7f8be80ccea0 2024-05-04T12:18:49.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 -- 172.21.15.52:0/1406702608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be80ccea0 msgr2=0x7f8be806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 --2- 172.21.15.52:0/1406702608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be80ccea0 0x7f8be806e080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8be8073670 tx=0x7f8be001c870 comp rx=0 tx=0).stop 2024-05-04T12:18:49.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 -- 172.21.15.52:0/1406702608 shutdown_connections 2024-05-04T12:18:49.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 --2- 172.21.15.52:0/1406702608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be80ccea0 0x7f8be806e080 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 --2- 172.21.15.52:0/1406702608 >> v2:172.21.15.111:3300/0 conn(0x7f8be8145960 0x7f8be8160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 -- 172.21.15.52:0/1406702608 >> 172.21.15.52:0/1406702608 conn(0x7f8be8067730 msgr2=0x7f8be80b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:49.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 -- 172.21.15.52:0/1406702608 shutdown_connections 2024-05-04T12:18:49.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 -- 172.21.15.52:0/1406702608 wait complete. 2024-05-04T12:18:49.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 Processor -- start 2024-05-04T12:18:49.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 -- start start 2024-05-04T12:18:49.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be80ccea0 0x7f8be8146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be8145960 0x7f8be813ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8be815c590 0x7f8be813f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8be806cdc0 con 0x7f8be80ccea0 2024-05-04T12:18:49.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8be806cc40 con 0x7f8be8145960 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f8be8145d40 con 0x7f8be815c590 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bedf01640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be80ccea0 0x7f8be8146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bedf01640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be80ccea0 0x7f8be8146040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53630/0 (socket says 172.21.15.52:53630) 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bedf01640 1 -- 172.21.15.52:0/3805253022 learned_addr learned my addr 172.21.15.52:0/3805253022 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bed700640 1 --2- 172.21.15.52:0/3805253022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be8145960 0x7f8be813ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bee702640 1 --2- 172.21.15.52:0/3805253022 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8be815c590 0x7f8be813f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bedf01640 1 -- 172.21.15.52:0/3805253022 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8be815c590 msgr2=0x7f8be813f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bedf01640 1 --2- 172.21.15.52:0/3805253022 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8be815c590 0x7f8be813f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bedf01640 1 -- 172.21.15.52:0/3805253022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be8145960 msgr2=0x7f8be813ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bedf01640 1 --2- 172.21.15.52:0/3805253022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be8145960 0x7f8be813ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bedf01640 1 -- 172.21.15.52:0/3805253022 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be8141930 con 0x7f8be80ccea0 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bedf01640 1 --2- 172.21.15.52:0/3805253022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be80ccea0 0x7f8be8146040 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8be400d800 tx=0x7f8be400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bde7fc640 1 -- 172.21.15.52:0/3805253022 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be400b6c0 con 0x7f8be80ccea0 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bde7fc640 1 -- 172.21.15.52:0/3805253022 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be4019070 con 0x7f8be80ccea0 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 -- 172.21.15.52:0/3805253022 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be813fa20 con 0x7f8be80ccea0 2024-05-04T12:18:49.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bde7fc640 1 -- 172.21.15.52:0/3805253022 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be400ba60 con 0x7f8be80ccea0 2024-05-04T12:18:49.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.489+0000 7f8bf0165640 1 -- 172.21.15.52:0/3805253022 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be8159df0 con 0x7f8be80ccea0 2024-05-04T12:18:49.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.493+0000 7f8bf0165640 1 -- 172.21.15.52:0/3805253022 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be8062e50 con 0x7f8be80ccea0 2024-05-04T12:18:49.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.493+0000 7f8bde7fc640 1 -- 172.21.15.52:0/3805253022 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8be400bc00 con 0x7f8be80ccea0 2024-05-04T12:18:49.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.493+0000 7f8bde7fc640 1 --2- 172.21.15.52:0/3805253022 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8bcc0678c0 0x7f8bcc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.493+0000 7f8bde7fc640 1 -- 172.21.15.52:0/3805253022 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8be4086200 con 0x7f8be80ccea0 2024-05-04T12:18:49.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.493+0000 7f8bed700640 1 --2- 172.21.15.52:0/3805253022 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8bcc0678c0 0x7f8bcc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.493+0000 7f8bed700640 1 --2- 172.21.15.52:0/3805253022 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8bcc0678c0 0x7f8bcc069d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8be815c460 tx=0x7f8bd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:49.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.497+0000 7f8bde7fc640 1 -- 172.21.15.52:0/3805253022 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8be813fa20 con 0x7f8be80ccea0 2024-05-04T12:18:49.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.657+0000 7f8bf0165640 1 -- 172.21.15.52:0/3805253022 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8be8075320 con 0x7f8be80ccea0 2024-05-04T12:18:49.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.657+0000 7f8bde7fc640 1 -- 172.21.15.52:0/3805253022 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f8be8141930 con 0x7f8be80ccea0 2024-05-04T12:18:49.662 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:18:49.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.661+0000 7f8bf0165640 1 -- 172.21.15.52:0/3805253022 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8bcc0678c0 msgr2=0x7f8bcc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.661+0000 7f8bf0165640 1 --2- 172.21.15.52:0/3805253022 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8bcc0678c0 0x7f8bcc069d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8be815c460 tx=0x7f8bd8009040 comp rx=0 tx=0).stop 2024-05-04T12:18:49.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.661+0000 7f8bf0165640 1 -- 172.21.15.52:0/3805253022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be80ccea0 msgr2=0x7f8be8146040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.661+0000 7f8bf0165640 1 --2- 172.21.15.52:0/3805253022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be80ccea0 0x7f8be8146040 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8be400d800 tx=0x7f8be400dcc0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.661+0000 7f8bf0165640 1 -- 172.21.15.52:0/3805253022 shutdown_connections 2024-05-04T12:18:49.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.661+0000 7f8bf0165640 1 --2- 172.21.15.52:0/3805253022 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8bcc0678c0 0x7f8bcc069d80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.661+0000 7f8bf0165640 1 --2- 172.21.15.52:0/3805253022 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8be815c590 0x7f8be813f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.661+0000 7f8bf0165640 1 --2- 172.21.15.52:0/3805253022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be8145960 0x7f8be813ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.661+0000 7f8bf0165640 1 --2- 172.21.15.52:0/3805253022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be80ccea0 0x7f8be8146040 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.661+0000 7f8bf0165640 1 -- 172.21.15.52:0/3805253022 >> 172.21.15.52:0/3805253022 conn(0x7f8be8067730 msgr2=0x7f8be8144460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:49.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.661+0000 7f8bf0165640 1 -- 172.21.15.52:0/3805253022 shutdown_connections 2024-05-04T12:18:49.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.661+0000 7f8bf0165640 1 -- 172.21.15.52:0/3805253022 wait complete. 2024-05-04T12:18:49.681 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:18:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.805+0000 7f6a77f42640 1 Processor -- start 2024-05-04T12:18:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.805+0000 7f6a77f42640 1 -- start start 2024-05-04T12:18:49.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.809+0000 7f6a77f42640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6a701428c0 0x7f6a70169140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.809+0000 7f6a77f42640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a700d2710 con 0x7f6a70144570 2024-05-04T12:18:49.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.809+0000 7f6a77f42640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a700d29c0 con 0x7f6a70141010 2024-05-04T12:18:49.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.809+0000 7f6a77f42640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f6a700d2ca0 con 0x7f6a701428c0 2024-05-04T12:18:49.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.809+0000 7f6a764df640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6a701428c0 0x7f6a70169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.809+0000 7f6a754dd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f6a70144570 0x7f6a70143940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55106/0 (socket says 172.21.15.52:55106) 2024-05-04T12:18:49.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.809+0000 7f6a754dd640 1 -- 172.21.15.52:0/3318005625 learned_addr learned my addr 172.21.15.52:0/3318005625 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:49.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.809+0000 7f6a764df640 1 -- 172.21.15.52:0/3318005625 >> v1:172.21.15.111:6789/0 conn(0x7f6a70141010 legacy=0x7f6a700d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:49.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.809+0000 7f6a764df640 1 -- 172.21.15.52:0/3318005625 >> v1:172.21.15.52:6789/0 conn(0x7f6a70144570 legacy=0x7f6a70143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.809+0000 7f6a764df640 1 -- 172.21.15.52:0/3318005625 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a700d2f80 con 0x7f6a701428c0 2024-05-04T12:18:49.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.809+0000 7f6a764df640 1 --2- 172.21.15.52:0/3318005625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a701428c0 0x7f6a70169140 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009d90 tx=0x7f6a6801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6259eeccf33359bd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:49.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.809+0000 7f6a74cdc640 1 -- 172.21.15.52:0/3318005625 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a680b2070 con 0x7f6a701428c0 2024-05-04T12:18:49.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.809+0000 7f6a74cdc640 1 -- 172.21.15.52:0/3318005625 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a680a5930 con 0x7f6a701428c0 2024-05-04T12:18:49.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.809+0000 7f6a74cdc640 1 -- 172.21.15.52:0/3318005625 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a680a5d00 con 0x7f6a701428c0 2024-05-04T12:18:49.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 -- 172.21.15.52:0/3318005625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a701428c0 msgr2=0x7f6a70169140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 --2- 172.21.15.52:0/3318005625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a701428c0 0x7f6a70169140 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009d90 tx=0x7f6a6801c870 comp rx=0 tx=0).stop 2024-05-04T12:18:49.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 -- 172.21.15.52:0/3318005625 shutdown_connections 2024-05-04T12:18:49.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 --2- 172.21.15.52:0/3318005625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a701428c0 0x7f6a70169140 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 -- 172.21.15.52:0/3318005625 >> 172.21.15.52:0/3318005625 conn(0x7f6a70072be0 msgr2=0x7f6a700d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:49.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 -- 172.21.15.52:0/3318005625 shutdown_connections 2024-05-04T12:18:49.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 -- 172.21.15.52:0/3318005625 wait complete. 2024-05-04T12:18:49.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 Processor -- start 2024-05-04T12:18:49.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 -- start start 2024-05-04T12:18:49.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a70141010 0x7f6a7013c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a701428c0 0x7f6a7015ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70144570 0x7f6a7015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6a70144050 con 0x7f6a701428c0 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6a70141600 con 0x7f6a70144570 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a77f42640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f6a701441d0 con 0x7f6a70141010 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a754dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a701428c0 0x7f6a7015ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a754dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a701428c0 0x7f6a7015ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53650/0 (socket says 172.21.15.52:53650) 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a754dd640 1 -- 172.21.15.52:0/2220112279 learned_addr learned my addr 172.21.15.52:0/2220112279 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a764df640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70144570 0x7f6a7015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a75cde640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a70141010 0x7f6a7013c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a754dd640 1 -- 172.21.15.52:0/2220112279 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a70141010 msgr2=0x7f6a7013c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a754dd640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a70141010 0x7f6a7013c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a754dd640 1 -- 172.21.15.52:0/2220112279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70144570 msgr2=0x7f6a7015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a754dd640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70144570 0x7f6a7015d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a754dd640 1 -- 172.21.15.52:0/2220112279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7015b780 con 0x7f6a701428c0 2024-05-04T12:18:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a75cde640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a70141010 0x7f6a7013c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a764df640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70144570 0x7f6a7015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a754dd640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a701428c0 0x7f6a7015ae50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000c340 tx=0x7f6a6000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.813+0000 7f6a667fc640 1 -- 172.21.15.52:0/2220112279 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60017070 con 0x7f6a701428c0 2024-05-04T12:18:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.817+0000 7f6a667fc640 1 -- 172.21.15.52:0/2220112279 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a60005cf0 con 0x7f6a701428c0 2024-05-04T12:18:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.817+0000 7f6a77f42640 1 -- 172.21.15.52:0/2220112279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a7015ba60 con 0x7f6a701428c0 2024-05-04T12:18:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.817+0000 7f6a667fc640 1 -- 172.21.15.52:0/2220112279 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a600072f0 con 0x7f6a701428c0 2024-05-04T12:18:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.817+0000 7f6a77f42640 1 -- 172.21.15.52:0/2220112279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a7013a9d0 con 0x7f6a701428c0 2024-05-04T12:18:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.817+0000 7f6a77f42640 1 -- 172.21.15.52:0/2220112279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a70062e50 con 0x7f6a701428c0 2024-05-04T12:18:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.817+0000 7f6a667fc640 1 -- 172.21.15.52:0/2220112279 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6a6001a070 con 0x7f6a701428c0 2024-05-04T12:18:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.817+0000 7f6a667fc640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6a540678c0 0x7f6a54069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.817+0000 7f6a667fc640 1 -- 172.21.15.52:0/2220112279 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6a60084fb0 con 0x7f6a701428c0 2024-05-04T12:18:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.817+0000 7f6a75cde640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6a540678c0 0x7f6a54069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.817+0000 7f6a75cde640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6a540678c0 0x7f6a54069d80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6a700ae790 tx=0x7f6a6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:49.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.821+0000 7f6a667fc640 1 -- 172.21.15.52:0/2220112279 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6a60055af0 con 0x7f6a701428c0 2024-05-04T12:18:49.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.973+0000 7f6a77f42640 1 -- 172.21.15.52:0/2220112279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6a700b2d10 con 0x7f6a701428c0 2024-05-04T12:18:49.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.973+0000 7f6a667fc640 1 -- 172.21.15.52:0/2220112279 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f6a600553f0 con 0x7f6a701428c0 2024-05-04T12:18:49.978 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:18:49.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.977+0000 7f6a77f42640 1 -- 172.21.15.52:0/2220112279 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6a540678c0 msgr2=0x7f6a54069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.977+0000 7f6a77f42640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6a540678c0 0x7f6a54069d80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6a700ae790 tx=0x7f6a6c009040 comp rx=0 tx=0).stop 2024-05-04T12:18:49.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.977+0000 7f6a77f42640 1 -- 172.21.15.52:0/2220112279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a701428c0 msgr2=0x7f6a7015ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:49.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.977+0000 7f6a77f42640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a701428c0 0x7f6a7015ae50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000c340 tx=0x7f6a6000c800 comp rx=0 tx=0).stop 2024-05-04T12:18:49.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.977+0000 7f6a77f42640 1 -- 172.21.15.52:0/2220112279 shutdown_connections 2024-05-04T12:18:49.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.977+0000 7f6a77f42640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6a540678c0 0x7f6a54069d80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.977+0000 7f6a77f42640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70144570 0x7f6a7015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.977+0000 7f6a77f42640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a701428c0 0x7f6a7015ae50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.977+0000 7f6a77f42640 1 --2- 172.21.15.52:0/2220112279 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a70141010 0x7f6a7013c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:49.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.977+0000 7f6a77f42640 1 -- 172.21.15.52:0/2220112279 >> 172.21.15.52:0/2220112279 conn(0x7f6a70072be0 msgr2=0x7f6a7015ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:49.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.977+0000 7f6a77f42640 1 -- 172.21.15.52:0/2220112279 shutdown_connections 2024-05-04T12:18:49.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:49.977+0000 7f6a77f42640 1 -- 172.21.15.52:0/2220112279 wait complete. 2024-05-04T12:18:49.996 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:50.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.105+0000 7f69070bd640 1 Processor -- start 2024-05-04T12:18:50.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.105+0000 7f69070bd640 1 -- start start 2024-05-04T12:18:50.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f69070bd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f690015e5f0 0x7f69000cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f69070bd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f690006b4b0 0x7f690006b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f69070bd640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f690015ef30 con 0x7f69000cd5f0 2024-05-04T12:18:50.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f69070bd640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f69000b2200 con 0x7f690006b4b0 2024-05-04T12:18:50.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f69070bd640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f690006be20 con 0x7f690015e5f0 2024-05-04T12:18:50.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f690565a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f690006b4b0 0x7f690006b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f690565a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f690006b4b0 0x7f690006b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53660/0 (socket says 172.21.15.52:53660) 2024-05-04T12:18:50.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f690565a640 1 -- 172.21.15.52:0/1350832403 learned_addr learned my addr 172.21.15.52:0/1350832403 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:50.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f6904e59640 1 --2- 172.21.15.52:0/1350832403 >> v2:172.21.15.111:3300/0 conn(0x7f690015e5f0 0x7f69000cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f690565a640 1 -- 172.21.15.52:0/1350832403 >> v2:172.21.15.111:3300/0 conn(0x7f690015e5f0 msgr2=0x7f69000cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:50.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f690565a640 1 --2- 172.21.15.52:0/1350832403 >> v2:172.21.15.111:3300/0 conn(0x7f690015e5f0 0x7f69000cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f690565a640 1 -- 172.21.15.52:0/1350832403 >> v1:172.21.15.52:6789/0 conn(0x7f69000cd5f0 legacy=0x7f690015f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f68ff7fe640 1 -- 172.21.15.52:0/1350832403 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1823920315 0 0) 0x7f690015ef30 con 0x7f69000cd5f0 2024-05-04T12:18:50.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f690565a640 1 -- 172.21.15.52:0/1350832403 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69000b1960 con 0x7f690006b4b0 2024-05-04T12:18:50.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f6904e59640 1 --2- 172.21.15.52:0/1350832403 >> v2:172.21.15.111:3300/0 conn(0x7f690015e5f0 0x7f69000cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:50.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f690565a640 1 --2- 172.21.15.52:0/1350832403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690006b4b0 0x7f690006b8e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f69000d1f70 tx=0x7f68f801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c404aef70f09323 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:50.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f68ff7fe640 1 -- 172.21.15.52:0/1350832403 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f800b2d0 con 0x7f690006b4b0 2024-05-04T12:18:50.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f68ff7fe640 1 -- 172.21.15.52:0/1350832403 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f800b470 con 0x7f690006b4b0 2024-05-04T12:18:50.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f68ff7fe640 1 -- 172.21.15.52:0/1350832403 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f80a7bd0 con 0x7f690006b4b0 2024-05-04T12:18:50.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f69070bd640 1 -- 172.21.15.52:0/1350832403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690006b4b0 msgr2=0x7f690006b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.109+0000 7f69070bd640 1 --2- 172.21.15.52:0/1350832403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690006b4b0 0x7f690006b8e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f69000d1f70 tx=0x7f68f801ebf0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 -- 172.21.15.52:0/1350832403 shutdown_connections 2024-05-04T12:18:50.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 --2- 172.21.15.52:0/1350832403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690006b4b0 0x7f690006b8e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 --2- 172.21.15.52:0/1350832403 >> v2:172.21.15.111:3300/0 conn(0x7f690015e5f0 0x7f69000cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 -- 172.21.15.52:0/1350832403 >> 172.21.15.52:0/1350832403 conn(0x7f69000b2380 msgr2=0x7f69000b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:50.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 -- 172.21.15.52:0/1350832403 shutdown_connections 2024-05-04T12:18:50.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 -- 172.21.15.52:0/1350832403 wait complete. 2024-05-04T12:18:50.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 Processor -- start 2024-05-04T12:18:50.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 -- start start 2024-05-04T12:18:50.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f690006b4b0 0x7f690014f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f69000cd5f0 0x7f6900072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690015e5f0 0x7f6900153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f690015eb50 con 0x7f690015e5f0 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f690015e9d0 con 0x7f690006b4b0 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f69000b1510 con 0x7f69000cd5f0 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f690565a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690015e5f0 0x7f6900153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f690565a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690015e5f0 0x7f6900153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53674/0 (socket says 172.21.15.52:53674) 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f690565a640 1 -- 172.21.15.52:0/202745504 learned_addr learned my addr 172.21.15.52:0/202745504 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f68fffff640 1 --2- 172.21.15.52:0/202745504 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f69000cd5f0 0x7f6900072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f6904e59640 1 --2- 172.21.15.52:0/202745504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f690006b4b0 0x7f690014f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f690565a640 1 -- 172.21.15.52:0/202745504 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f69000cd5f0 msgr2=0x7f6900072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f690565a640 1 --2- 172.21.15.52:0/202745504 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f69000cd5f0 0x7f6900072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f690565a640 1 -- 172.21.15.52:0/202745504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f690006b4b0 msgr2=0x7f690014f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f690565a640 1 --2- 172.21.15.52:0/202745504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f690006b4b0 0x7f690014f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f690565a640 1 -- 172.21.15.52:0/202745504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69001525f0 con 0x7f690015e5f0 2024-05-04T12:18:50.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f690565a640 1 --2- 172.21.15.52:0/202745504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690015e5f0 0x7f6900153480 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f68f801b800 tx=0x7f68f801b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:50.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f68fd7fa640 1 -- 172.21.15.52:0/202745504 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f80be030 con 0x7f690015e5f0 2024-05-04T12:18:50.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f68fd7fa640 1 -- 172.21.15.52:0/202745504 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f8009600 con 0x7f690015e5f0 2024-05-04T12:18:50.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 -- 172.21.15.52:0/202745504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69001528d0 con 0x7f690015e5f0 2024-05-04T12:18:50.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f68fd7fa640 1 -- 172.21.15.52:0/202745504 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f80099d0 con 0x7f690015e5f0 2024-05-04T12:18:50.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.113+0000 7f69070bd640 1 -- 172.21.15.52:0/202745504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6900070980 con 0x7f690015e5f0 2024-05-04T12:18:50.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.117+0000 7f69070bd640 1 -- 172.21.15.52:0/202745504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6900062e50 con 0x7f690015e5f0 2024-05-04T12:18:50.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.117+0000 7f68fd7fa640 1 -- 172.21.15.52:0/202745504 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f68f80c1070 con 0x7f690015e5f0 2024-05-04T12:18:50.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.117+0000 7f68fd7fa640 1 --2- 172.21.15.52:0/202745504 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f68ec0678c0 0x7f68ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.117+0000 7f68fd7fa640 1 -- 172.21.15.52:0/202745504 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f68f8122af0 con 0x7f690015e5f0 2024-05-04T12:18:50.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.117+0000 7f6904e59640 1 --2- 172.21.15.52:0/202745504 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f68ec0678c0 0x7f68ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.117+0000 7f6904e59640 1 --2- 172.21.15.52:0/202745504 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f68ec0678c0 0x7f68ec069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f68f4006fd0 tx=0x7f68f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:50.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.121+0000 7f68fd7fa640 1 -- 172.21.15.52:0/202745504 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f68f80187e0 con 0x7f690015e5f0 2024-05-04T12:18:50.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.285+0000 7f69070bd640 1 -- 172.21.15.52:0/202745504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6900165ee0 con 0x7f690015e5f0 2024-05-04T12:18:50.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.285+0000 7f68fd7fa640 1 -- 172.21.15.52:0/202745504 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f68f80f36c0 con 0x7f690015e5f0 2024-05-04T12:18:50.290 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:50.290 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:50.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.289+0000 7f69070bd640 1 -- 172.21.15.52:0/202745504 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f68ec0678c0 msgr2=0x7f68ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.289+0000 7f69070bd640 1 --2- 172.21.15.52:0/202745504 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f68ec0678c0 0x7f68ec069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f68f4006fd0 tx=0x7f68f4008040 comp rx=0 tx=0).stop 2024-05-04T12:18:50.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.289+0000 7f69070bd640 1 -- 172.21.15.52:0/202745504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690015e5f0 msgr2=0x7f6900153480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.289+0000 7f69070bd640 1 --2- 172.21.15.52:0/202745504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690015e5f0 0x7f6900153480 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f68f801b800 tx=0x7f68f801b990 comp rx=0 tx=0).stop 2024-05-04T12:18:50.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.289+0000 7f69070bd640 1 -- 172.21.15.52:0/202745504 shutdown_connections 2024-05-04T12:18:50.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.289+0000 7f69070bd640 1 --2- 172.21.15.52:0/202745504 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f68ec0678c0 0x7f68ec069d80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.289+0000 7f69070bd640 1 --2- 172.21.15.52:0/202745504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690015e5f0 0x7f6900153480 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.289+0000 7f69070bd640 1 --2- 172.21.15.52:0/202745504 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f69000cd5f0 0x7f6900072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.289+0000 7f69070bd640 1 --2- 172.21.15.52:0/202745504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f690006b4b0 0x7f690014f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.289+0000 7f69070bd640 1 -- 172.21.15.52:0/202745504 >> 172.21.15.52:0/202745504 conn(0x7f69000b2380 msgr2=0x7f690006dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:50.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.289+0000 7f69070bd640 1 -- 172.21.15.52:0/202745504 shutdown_connections 2024-05-04T12:18:50.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.289+0000 7f69070bd640 1 -- 172.21.15.52:0/202745504 wait complete. 2024-05-04T12:18:50.309 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:18:50.310 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:18:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.441+0000 7f4eaabfb640 1 Processor -- start 2024-05-04T12:18:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4eaabfb640 1 -- start start 2024-05-04T12:18:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4eaabfb640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4ea40c4630 0x7f4ea40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4eaabfb640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ea4070c10 0x7f4ea40c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4eaabfb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4ea40c52b0 0x7f4ea40cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4eaabfb640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea4071070 con 0x7f4ea40c4630 2024-05-04T12:18:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4eaabfb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea40c5a80 con 0x7f4ea40c52b0 2024-05-04T12:18:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4eaabfb640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea415c470 con 0x7f4ea4070c10 2024-05-04T12:18:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4ea8997640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4ea40c4630 0x7f4ea40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4ea8997640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4ea40c4630 0x7f4ea40706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53680/0 (socket says 172.21.15.52:53680) 2024-05-04T12:18:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4ea8997640 1 -- 172.21.15.52:0/2519775990 learned_addr learned my addr 172.21.15.52:0/2519775990 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4ea3fff640 1 --2- 172.21.15.52:0/2519775990 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ea4070c10 0x7f4ea40c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4ea9198640 1 --2- 172.21.15.52:0/2519775990 >> v2:172.21.15.111:3300/0 conn(0x7f4ea40c52b0 0x7f4ea40cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4ea8997640 1 -- 172.21.15.52:0/2519775990 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ea4070c10 msgr2=0x7f4ea40c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4ea8997640 1 --2- 172.21.15.52:0/2519775990 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ea4070c10 0x7f4ea40c4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4ea8997640 1 -- 172.21.15.52:0/2519775990 >> v2:172.21.15.111:3300/0 conn(0x7f4ea40c52b0 msgr2=0x7f4ea40cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4ea8997640 1 --2- 172.21.15.52:0/2519775990 >> v2:172.21.15.111:3300/0 conn(0x7f4ea40c52b0 0x7f4ea40cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4ea8997640 1 -- 172.21.15.52:0/2519775990 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea415c6c0 con 0x7f4ea40c4630 2024-05-04T12:18:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4ea8997640 1 --2- 172.21.15.52:0/2519775990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea40c4630 0x7f4ea40706d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4ea415f9a0 tx=0x7f4e9c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6c1f4b9545b246 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4ea37fe640 1 -- 172.21.15.52:0/2519775990 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0b0070 con 0x7f4ea40c4630 2024-05-04T12:18:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4ea37fe640 1 -- 172.21.15.52:0/2519775990 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e9c0ab600 con 0x7f4ea40c4630 2024-05-04T12:18:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.445+0000 7f4ea37fe640 1 -- 172.21.15.52:0/2519775990 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0ab9d0 con 0x7f4ea40c4630 2024-05-04T12:18:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 -- 172.21.15.52:0/2519775990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea40c4630 msgr2=0x7f4ea40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 --2- 172.21.15.52:0/2519775990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea40c4630 0x7f4ea40706d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4ea415f9a0 tx=0x7f4e9c01c260 comp rx=0 tx=0).stop 2024-05-04T12:18:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 -- 172.21.15.52:0/2519775990 shutdown_connections 2024-05-04T12:18:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 --2- 172.21.15.52:0/2519775990 >> v2:172.21.15.111:3300/0 conn(0x7f4ea40c52b0 0x7f4ea40cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 --2- 172.21.15.52:0/2519775990 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ea4070c10 0x7f4ea40c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 --2- 172.21.15.52:0/2519775990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea40c4630 0x7f4ea40706d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 -- 172.21.15.52:0/2519775990 >> 172.21.15.52:0/2519775990 conn(0x7f4ea40d12c0 msgr2=0x7f4ea40d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 -- 172.21.15.52:0/2519775990 shutdown_connections 2024-05-04T12:18:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 -- 172.21.15.52:0/2519775990 wait complete. 2024-05-04T12:18:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 Processor -- start 2024-05-04T12:18:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 -- start start 2024-05-04T12:18:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ea4070c10 0x7f4ea413ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea40c4630 0x7f4ea4072c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ea40c52b0 0x7f4ea4073190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea40c5840 con 0x7f4ea40c4630 2024-05-04T12:18:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea40c56c0 con 0x7f4ea40c52b0 2024-05-04T12:18:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea40c3b70 con 0x7f4ea4070c10 2024-05-04T12:18:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4ea3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea40c4630 0x7f4ea4072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4ea3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea40c4630 0x7f4ea4072c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53696/0 (socket says 172.21.15.52:53696) 2024-05-04T12:18:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4ea3fff640 1 -- 172.21.15.52:0/1109052590 learned_addr learned my addr 172.21.15.52:0/1109052590 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4ea8997640 1 --2- 172.21.15.52:0/1109052590 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ea4070c10 0x7f4ea413ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4ea9198640 1 --2- 172.21.15.52:0/1109052590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ea40c52b0 0x7f4ea4073190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4ea3fff640 1 -- 172.21.15.52:0/1109052590 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ea4070c10 msgr2=0x7f4ea413ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4ea3fff640 1 --2- 172.21.15.52:0/1109052590 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ea4070c10 0x7f4ea413ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4ea3fff640 1 -- 172.21.15.52:0/1109052590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ea40c52b0 msgr2=0x7f4ea4073190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4ea3fff640 1 --2- 172.21.15.52:0/1109052590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ea40c52b0 0x7f4ea4073190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4ea3fff640 1 -- 172.21.15.52:0/1109052590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea414c040 con 0x7f4ea40c4630 2024-05-04T12:18:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4ea3fff640 1 --2- 172.21.15.52:0/1109052590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea40c4630 0x7f4ea4072c50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800d800 tx=0x7f4e9800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4ea17fa640 1 -- 172.21.15.52:0/1109052590 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800b6c0 con 0x7f4ea40c4630 2024-05-04T12:18:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4ea17fa640 1 -- 172.21.15.52:0/1109052590 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e98019070 con 0x7f4ea40c4630 2024-05-04T12:18:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4ea17fa640 1 -- 172.21.15.52:0/1109052590 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800ba60 con 0x7f4ea40c4630 2024-05-04T12:18:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 -- 172.21.15.52:0/1109052590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea413f140 con 0x7f4ea40c4630 2024-05-04T12:18:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.449+0000 7f4eaabfb640 1 -- 172.21.15.52:0/1109052590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea4153300 con 0x7f4ea40c4630 2024-05-04T12:18:50.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.453+0000 7f4eaabfb640 1 -- 172.21.15.52:0/1109052590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea4062e50 con 0x7f4ea40c4630 2024-05-04T12:18:50.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.453+0000 7f4ea17fa640 1 -- 172.21.15.52:0/1109052590 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4e9800bc00 con 0x7f4ea40c4630 2024-05-04T12:18:50.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.453+0000 7f4ea17fa640 1 --2- 172.21.15.52:0/1109052590 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4e900678c0 0x7f4e90069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.453+0000 7f4ea8997640 1 --2- 172.21.15.52:0/1109052590 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4e900678c0 0x7f4e90069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.453+0000 7f4ea17fa640 1 -- 172.21.15.52:0/1109052590 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4e98086140 con 0x7f4ea40c4630 2024-05-04T12:18:50.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.453+0000 7f4ea8997640 1 --2- 172.21.15.52:0/1109052590 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4e900678c0 0x7f4e90069d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0ad000 tx=0x7f4e9c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:50.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.457+0000 7f4ea17fa640 1 -- 172.21.15.52:0/1109052590 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4ea414c040 con 0x7f4ea40c4630 2024-05-04T12:18:50.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.617+0000 7f4eaabfb640 1 -- 172.21.15.52:0/1109052590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4ea406e8c0 con 0x7f4ea40c4630 2024-05-04T12:18:50.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.621+0000 7f4ea17fa640 1 -- 172.21.15.52:0/1109052590 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f4e98056c70 con 0x7f4ea40c4630 2024-05-04T12:18:50.623 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:18:50.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.621+0000 7f4eaabfb640 1 -- 172.21.15.52:0/1109052590 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4e900678c0 msgr2=0x7f4e90069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.621+0000 7f4eaabfb640 1 --2- 172.21.15.52:0/1109052590 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4e900678c0 0x7f4e90069d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0ad000 tx=0x7f4e9c0a6000 comp rx=0 tx=0).stop 2024-05-04T12:18:50.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.621+0000 7f4eaabfb640 1 -- 172.21.15.52:0/1109052590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea40c4630 msgr2=0x7f4ea4072c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.621+0000 7f4eaabfb640 1 --2- 172.21.15.52:0/1109052590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea40c4630 0x7f4ea4072c50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800d800 tx=0x7f4e9800dcc0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.621+0000 7f4eaabfb640 1 -- 172.21.15.52:0/1109052590 shutdown_connections 2024-05-04T12:18:50.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.621+0000 7f4eaabfb640 1 --2- 172.21.15.52:0/1109052590 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4e900678c0 0x7f4e90069d80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.621+0000 7f4eaabfb640 1 --2- 172.21.15.52:0/1109052590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ea40c52b0 0x7f4ea4073190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.621+0000 7f4eaabfb640 1 --2- 172.21.15.52:0/1109052590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea40c4630 0x7f4ea4072c50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.621+0000 7f4eaabfb640 1 --2- 172.21.15.52:0/1109052590 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ea4070c10 0x7f4ea413ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.621+0000 7f4eaabfb640 1 -- 172.21.15.52:0/1109052590 >> 172.21.15.52:0/1109052590 conn(0x7f4ea40d12c0 msgr2=0x7f4ea4072220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:50.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.621+0000 7f4eaabfb640 1 -- 172.21.15.52:0/1109052590 shutdown_connections 2024-05-04T12:18:50.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.621+0000 7f4eaabfb640 1 -- 172.21.15.52:0/1109052590 wait complete. 2024-05-04T12:18:50.643 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:18:50.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd68cbc2640 1 Processor -- start 2024-05-04T12:18:50.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd68cbc2640 1 -- start start 2024-05-04T12:18:50.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd68cbc2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd6880c4630 0x7fd6880706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd68cbc2640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd6880c5480 0x7fd68815fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd68cbc2640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd68815c6c0 con 0x7fd688070c10 2024-05-04T12:18:50.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd68cbc2640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fd688071070 con 0x7fd6880c5480 2024-05-04T12:18:50.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd68cbc2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fd6880c5a80 con 0x7fd6880c4630 2024-05-04T12:18:50.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd686d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd6880c5480 0x7fd68815fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd685d9b640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fd688070c10 0x7fd6880c4d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55130/0 (socket says 172.21.15.52:55130) 2024-05-04T12:18:50.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd685d9b640 1 -- 172.21.15.52:0/2043056524 learned_addr learned my addr 172.21.15.52:0/2043056524 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:50.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd68659c640 1 --2- 172.21.15.52:0/2043056524 >> v2:172.21.15.111:3300/0 conn(0x7fd6880c4630 0x7fd6880706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd68559a640 1 -- 172.21.15.52:0/2043056524 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2266219756 0 0) 0x7fd68815c6c0 con 0x7fd688070c10 2024-05-04T12:18:50.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd68559a640 1 -- 172.21.15.52:0/2043056524 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd670003530 con 0x7fd688070c10 2024-05-04T12:18:50.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd686d9d640 1 -- 172.21.15.52:0/2043056524 >> v2:172.21.15.111:3300/0 conn(0x7fd6880c4630 msgr2=0x7fd6880706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:50.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd686d9d640 1 --2- 172.21.15.52:0/2043056524 >> v2:172.21.15.111:3300/0 conn(0x7fd6880c4630 0x7fd6880706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd686d9d640 1 -- 172.21.15.52:0/2043056524 >> v1:172.21.15.52:6789/0 conn(0x7fd688070c10 legacy=0x7fd6880c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd686d9d640 1 -- 172.21.15.52:0/2043056524 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68815c9a0 con 0x7fd6880c5480 2024-05-04T12:18:50.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.757+0000 7fd68659c640 1 --2- 172.21.15.52:0/2043056524 >> v2:172.21.15.111:3300/0 conn(0x7fd6880c4630 0x7fd6880706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:50.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd686d9d640 1 --2- 172.21.15.52:0/2043056524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6880c5480 0x7fd68815fa60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd68806e440 tx=0x7fd68001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f791cab72ae44fa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:50.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68559a640 1 -- 172.21.15.52:0/2043056524 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6800b2070 con 0x7fd6880c5480 2024-05-04T12:18:50.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68559a640 1 -- 172.21.15.52:0/2043056524 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6800a5900 con 0x7fd6880c5480 2024-05-04T12:18:50.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68559a640 1 -- 172.21.15.52:0/2043056524 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6800a5cd0 con 0x7fd6880c5480 2024-05-04T12:18:50.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 -- 172.21.15.52:0/2043056524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6880c5480 msgr2=0x7fd68815fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 --2- 172.21.15.52:0/2043056524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6880c5480 0x7fd68815fa60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd68806e440 tx=0x7fd68001c930 comp rx=0 tx=0).stop 2024-05-04T12:18:50.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 -- 172.21.15.52:0/2043056524 shutdown_connections 2024-05-04T12:18:50.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 --2- 172.21.15.52:0/2043056524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6880c5480 0x7fd68815fa60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 --2- 172.21.15.52:0/2043056524 >> v2:172.21.15.111:3300/0 conn(0x7fd6880c4630 0x7fd6880706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 -- 172.21.15.52:0/2043056524 >> 172.21.15.52:0/2043056524 conn(0x7fd6880d12c0 msgr2=0x7fd6880d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:50.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 -- 172.21.15.52:0/2043056524 shutdown_connections 2024-05-04T12:18:50.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 -- 172.21.15.52:0/2043056524 wait complete. 2024-05-04T12:18:50.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 Processor -- start 2024-05-04T12:18:50.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 -- start start 2024-05-04T12:18:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd688070c10 0x7fd688151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6880c4630 0x7fd688157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6880c5480 0x7fd68814a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd68815cff0 con 0x7fd6880c4630 2024-05-04T12:18:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd688168e00 con 0x7fd688070c10 2024-05-04T12:18:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68cbc2640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fd688168f80 con 0x7fd6880c5480 2024-05-04T12:18:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd685d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6880c4630 0x7fd688157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd685d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6880c4630 0x7fd688157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53722/0 (socket says 172.21.15.52:53722) 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd685d9b640 1 -- 172.21.15.52:0/3320122898 learned_addr learned my addr 172.21.15.52:0/3320122898 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd686d9d640 1 --2- 172.21.15.52:0/3320122898 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6880c5480 0x7fd68814a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd68659c640 1 --2- 172.21.15.52:0/3320122898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd688070c10 0x7fd688151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd685d9b640 1 -- 172.21.15.52:0/3320122898 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6880c5480 msgr2=0x7fd68814a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd685d9b640 1 --2- 172.21.15.52:0/3320122898 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6880c5480 0x7fd68814a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd685d9b640 1 -- 172.21.15.52:0/3320122898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd688070c10 msgr2=0x7fd688151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd685d9b640 1 --2- 172.21.15.52:0/3320122898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd688070c10 0x7fd688151840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd685d9b640 1 -- 172.21.15.52:0/3320122898 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd688158550 con 0x7fd6880c4630 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.761+0000 7fd685d9b640 1 --2- 172.21.15.52:0/3320122898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6880c4630 0x7fd688157540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00c450 tx=0x7fd67c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.765+0000 7fd676ffd640 1 -- 172.21.15.52:0/3320122898 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c017070 con 0x7fd6880c4630 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.765+0000 7fd676ffd640 1 -- 172.21.15.52:0/3320122898 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd67c005cf0 con 0x7fd6880c4630 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.765+0000 7fd68cbc2640 1 -- 172.21.15.52:0/3320122898 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd688157cb0 con 0x7fd6880c4630 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.765+0000 7fd676ffd640 1 -- 172.21.15.52:0/3320122898 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c007290 con 0x7fd6880c4630 2024-05-04T12:18:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.765+0000 7fd68cbc2640 1 -- 172.21.15.52:0/3320122898 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd688158270 con 0x7fd6880c4630 2024-05-04T12:18:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.765+0000 7fd68cbc2640 1 -- 172.21.15.52:0/3320122898 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd688062e50 con 0x7fd6880c4630 2024-05-04T12:18:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.765+0000 7fd676ffd640 1 -- 172.21.15.52:0/3320122898 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd67c01a070 con 0x7fd6880c4630 2024-05-04T12:18:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.765+0000 7fd676ffd640 1 --2- 172.21.15.52:0/3320122898 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd6680678f0 0x7fd668069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.765+0000 7fd676ffd640 1 -- 172.21.15.52:0/3320122898 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd67c085020 con 0x7fd6880c4630 2024-05-04T12:18:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.765+0000 7fd68659c640 1 --2- 172.21.15.52:0/3320122898 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd6680678f0 0x7fd668069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.765+0000 7fd68659c640 1 --2- 172.21.15.52:0/3320122898 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd6680678f0 0x7fd668069db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd678006fd0 tx=0x7fd678008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:50.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.769+0000 7fd676ffd640 1 -- 172.21.15.52:0/3320122898 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd67c055b60 con 0x7fd6880c4630 2024-05-04T12:18:50.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.925+0000 7fd68cbc2640 1 -- 172.21.15.52:0/3320122898 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd6880c5eb0 con 0x7fd6880c4630 2024-05-04T12:18:50.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.925+0000 7fd676ffd640 1 -- 172.21.15.52:0/3320122898 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fd67c055460 con 0x7fd6880c4630 2024-05-04T12:18:50.927 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:18:50.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.925+0000 7fd68cbc2640 1 -- 172.21.15.52:0/3320122898 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd6680678f0 msgr2=0x7fd668069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.925+0000 7fd68cbc2640 1 --2- 172.21.15.52:0/3320122898 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd6680678f0 0x7fd668069db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd678006fd0 tx=0x7fd678008040 comp rx=0 tx=0).stop 2024-05-04T12:18:50.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.925+0000 7fd68cbc2640 1 -- 172.21.15.52:0/3320122898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6880c4630 msgr2=0x7fd688157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:50.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.925+0000 7fd68cbc2640 1 --2- 172.21.15.52:0/3320122898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6880c4630 0x7fd688157540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00c450 tx=0x7fd67c00c910 comp rx=0 tx=0).stop 2024-05-04T12:18:50.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.925+0000 7fd68cbc2640 1 -- 172.21.15.52:0/3320122898 shutdown_connections 2024-05-04T12:18:50.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.925+0000 7fd68cbc2640 1 --2- 172.21.15.52:0/3320122898 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd6680678f0 0x7fd668069db0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.925+0000 7fd68cbc2640 1 --2- 172.21.15.52:0/3320122898 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6880c5480 0x7fd68814a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.925+0000 7fd68cbc2640 1 --2- 172.21.15.52:0/3320122898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6880c4630 0x7fd688157540 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.925+0000 7fd68cbc2640 1 --2- 172.21.15.52:0/3320122898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd688070c10 0x7fd688151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:50.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.925+0000 7fd68cbc2640 1 -- 172.21.15.52:0/3320122898 >> 172.21.15.52:0/3320122898 conn(0x7fd6880d12c0 msgr2=0x7fd688071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:50.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.925+0000 7fd68cbc2640 1 -- 172.21.15.52:0/3320122898 shutdown_connections 2024-05-04T12:18:50.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:50.925+0000 7fd68cbc2640 1 -- 172.21.15.52:0/3320122898 wait complete. 2024-05-04T12:18:50.944 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:18:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f1661bdf640 1 Processor -- start 2024-05-04T12:18:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f1661bdf640 1 -- start start 2024-05-04T12:18:51.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f1661bdf640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f165c141010 0x7f165c0d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f1661bdf640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f165c0d2540 0x7f165c1418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f1661bdf640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f165c0d29c0 con 0x7f165c144570 2024-05-04T12:18:51.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f1661bdf640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f165c0d2ca0 con 0x7f165c141010 2024-05-04T12:18:51.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f1661bdf640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f165c0d2f80 con 0x7f165c0d2540 2024-05-04T12:18:51.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f165bfff640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f165c0d2540 0x7f165c1418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f165b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f165c141010 0x7f165c0d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f165affd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f165c144570 0x7f165c143940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.52:43792/0 (socket says 172.21.15.52:43792) 2024-05-04T12:18:51.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f165affd640 1 -- 172.21.15.52:0/4234288522 learned_addr learned my addr 172.21.15.52:0/4234288522 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:51.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f165b7fe640 1 -- 172.21.15.52:0/4234288522 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f165c0d2540 msgr2=0x7f165c1418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:51.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f165b7fe640 1 --2- 172.21.15.52:0/4234288522 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f165c0d2540 0x7f165c1418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f165b7fe640 1 -- 172.21.15.52:0/4234288522 >> v1:172.21.15.111:6789/0 conn(0x7f165c144570 legacy=0x7f165c143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f165b7fe640 1 -- 172.21.15.52:0/4234288522 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165c0d3260 con 0x7f165c141010 2024-05-04T12:18:51.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.057+0000 7f165bfff640 1 --2- 172.21.15.52:0/4234288522 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f165c0d2540 0x7f165c1418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:18:51.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.061+0000 7f165b7fe640 1 --2- 172.21.15.52:0/4234288522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f165c141010 0x7f165c0d2000 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f164c0097c0 tx=0x7f164c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9b5e3ddc23143db server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.061+0000 7f165a7fc640 1 -- 172.21.15.52:0/4234288522 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c0ab600 con 0x7f165c141010 2024-05-04T12:18:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.061+0000 7f165a7fc640 1 -- 172.21.15.52:0/4234288522 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f164c0b0070 con 0x7f165c141010 2024-05-04T12:18:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.061+0000 7f165a7fc640 1 -- 172.21.15.52:0/4234288522 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c0ab9c0 con 0x7f165c141010 2024-05-04T12:18:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 -- 172.21.15.52:0/4234288522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f165c141010 msgr2=0x7f165c0d2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 --2- 172.21.15.52:0/4234288522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f165c141010 0x7f165c0d2000 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f164c0097c0 tx=0x7f164c01c3e0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 -- 172.21.15.52:0/4234288522 shutdown_connections 2024-05-04T12:18:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 --2- 172.21.15.52:0/4234288522 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f165c0d2540 0x7f165c1418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 --2- 172.21.15.52:0/4234288522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f165c141010 0x7f165c0d2000 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 -- 172.21.15.52:0/4234288522 >> 172.21.15.52:0/4234288522 conn(0x7f165c072be0 msgr2=0x7f165c0d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 -- 172.21.15.52:0/4234288522 shutdown_connections 2024-05-04T12:18:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 -- 172.21.15.52:0/4234288522 wait complete. 2024-05-04T12:18:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 Processor -- start 2024-05-04T12:18:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 -- start start 2024-05-04T12:18:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f165c0d2540 0x7f165c13ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f165c141010 0x7f165c152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f165c144570 0x7f165c158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f165c142600 con 0x7f165c0d2540 2024-05-04T12:18:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f165c144260 con 0x7f165c141010 2024-05-04T12:18:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f165c142780 con 0x7f165c144570 2024-05-04T12:18:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f165b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f165c0d2540 0x7f165c13ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f165b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f165c0d2540 0x7f165c13ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53730/0 (socket says 172.21.15.52:53730) 2024-05-04T12:18:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f165b7fe640 1 -- 172.21.15.52:0/1858168593 learned_addr learned my addr 172.21.15.52:0/1858168593 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f165bfff640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f165c144570 0x7f165c158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f165affd640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f165c141010 0x7f165c152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f165b7fe640 1 -- 172.21.15.52:0/1858168593 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f165c144570 msgr2=0x7f165c158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f165b7fe640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f165c144570 0x7f165c158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f165b7fe640 1 -- 172.21.15.52:0/1858168593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f165c141010 msgr2=0x7f165c152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f165b7fe640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f165c141010 0x7f165c152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f165b7fe640 1 -- 172.21.15.52:0/1858168593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165c1597f0 con 0x7f165c0d2540 2024-05-04T12:18:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f165affd640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f165c141010 0x7f165c152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f165b7fe640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f165c0d2540 0x7f165c13ec40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f164c0097c0 tx=0x7f164c01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1643fff640 1 -- 172.21.15.52:0/1858168593 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c006e20 con 0x7f165c0d2540 2024-05-04T12:18:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1643fff640 1 -- 172.21.15.52:0/1858168593 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165c141600 con 0x7f165c0d2540 2024-05-04T12:18:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 -- 172.21.15.52:0/1858168593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165c159ad0 con 0x7f165c0d2540 2024-05-04T12:18:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1643fff640 1 -- 172.21.15.52:0/1858168593 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c0bf870 con 0x7f165c0d2540 2024-05-04T12:18:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.065+0000 7f1661bdf640 1 -- 172.21.15.52:0/1858168593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f165c15a090 con 0x7f165c0d2540 2024-05-04T12:18:51.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.069+0000 7f1661bdf640 1 -- 172.21.15.52:0/1858168593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f165c062e50 con 0x7f165c0d2540 2024-05-04T12:18:51.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.069+0000 7f1643fff640 1 -- 172.21.15.52:0/1858168593 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f164c0bd020 con 0x7f165c0d2540 2024-05-04T12:18:51.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.069+0000 7f1643fff640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f16480678f0 0x7f1648069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.069+0000 7f165affd640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f16480678f0 0x7f1648069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.069+0000 7f1643fff640 1 -- 172.21.15.52:0/1858168593 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f164c120010 con 0x7f165c0d2540 2024-05-04T12:18:51.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.069+0000 7f165affd640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f16480678f0 0x7f1648069db0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f165c0d15f0 tx=0x7f1650009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:51.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.073+0000 7f1643fff640 1 -- 172.21.15.52:0/1858168593 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f164c016090 con 0x7f165c0d2540 2024-05-04T12:18:51.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.225+0000 7f1661bdf640 1 -- 172.21.15.52:0/1858168593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f165c075320 con 0x7f165c0d2540 2024-05-04T12:18:51.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.225+0000 7f1643fff640 1 -- 172.21.15.52:0/1858168593 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f164c0f0b50 con 0x7f165c0d2540 2024-05-04T12:18:51.229 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:18:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.229+0000 7f1661bdf640 1 -- 172.21.15.52:0/1858168593 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f16480678f0 msgr2=0x7f1648069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.229+0000 7f1661bdf640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f16480678f0 0x7f1648069db0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f165c0d15f0 tx=0x7f1650009040 comp rx=0 tx=0).stop 2024-05-04T12:18:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.229+0000 7f1661bdf640 1 -- 172.21.15.52:0/1858168593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f165c0d2540 msgr2=0x7f165c13ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.229+0000 7f1661bdf640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f165c0d2540 0x7f165c13ec40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f164c0097c0 tx=0x7f164c01ac20 comp rx=0 tx=0).stop 2024-05-04T12:18:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.229+0000 7f1661bdf640 1 -- 172.21.15.52:0/1858168593 shutdown_connections 2024-05-04T12:18:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.229+0000 7f1661bdf640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f16480678f0 0x7f1648069db0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.229+0000 7f1661bdf640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f165c144570 0x7f165c158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.229+0000 7f1661bdf640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f165c141010 0x7f165c152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.229+0000 7f1661bdf640 1 --2- 172.21.15.52:0/1858168593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f165c0d2540 0x7f165c13ec40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.229+0000 7f1661bdf640 1 -- 172.21.15.52:0/1858168593 >> 172.21.15.52:0/1858168593 conn(0x7f165c072be0 msgr2=0x7f165c0d4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.229+0000 7f1661bdf640 1 -- 172.21.15.52:0/1858168593 shutdown_connections 2024-05-04T12:18:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.229+0000 7f1661bdf640 1 -- 172.21.15.52:0/1858168593 wait complete. 2024-05-04T12:18:51.247 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.357+0000 7f078eace640 1 Processor -- start 2024-05-04T12:18:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.357+0000 7f078eace640 1 -- start start 2024-05-04T12:18:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.357+0000 7f078eace640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f07880aa0c0 0x7f07880ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.357+0000 7f078eace640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f078806d1b0 con 0x7f07880aa550 2024-05-04T12:18:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.357+0000 7f078eace640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f078806d490 con 0x7f078815bf50 2024-05-04T12:18:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.357+0000 7f078eace640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f078815ca40 con 0x7f07880aa0c0 2024-05-04T12:18:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.357+0000 7f078d06b640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f07880aa550 0x7f078806e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55138/0 (socket says 172.21.15.52:55138) 2024-05-04T12:18:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.357+0000 7f078d06b640 1 -- 172.21.15.52:0/2417389812 learned_addr learned my addr 172.21.15.52:0/2417389812 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.357+0000 7f0787fff640 1 --2- 172.21.15.52:0/2417389812 >> v2:172.21.15.52:3300/0 conn(0x7f07880aa0c0 0x7f07880ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.357+0000 7f0787fff640 1 -- 172.21.15.52:0/2417389812 >> v1:172.21.15.111:6789/0 conn(0x7f078815bf50 legacy=0x7f078815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:51.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.357+0000 7f0787fff640 1 -- 172.21.15.52:0/2417389812 >> v1:172.21.15.52:6789/0 conn(0x7f07880aa550 legacy=0x7f078806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.357+0000 7f0787fff640 1 -- 172.21.15.52:0/2417389812 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078806d770 con 0x7f07880aa0c0 2024-05-04T12:18:51.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.357+0000 7f07877fe640 1 -- 172.21.15.52:0/2417389812 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2984948611 0 0) 0x7f078806d1b0 con 0x7f07880aa550 2024-05-04T12:18:51.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f0787fff640 1 --2- 172.21.15.52:0/2417389812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07880aa0c0 0x7f07880ccb50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f07740097c0 tx=0x7f077401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a789f98609e0c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:51.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f07877fe640 1 -- 172.21.15.52:0/2417389812 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07740ab650 con 0x7f07880aa0c0 2024-05-04T12:18:51.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f07877fe640 1 -- 172.21.15.52:0/2417389812 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07740b0070 con 0x7f07880aa0c0 2024-05-04T12:18:51.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f07877fe640 1 -- 172.21.15.52:0/2417389812 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07740aba10 con 0x7f07880aa0c0 2024-05-04T12:18:51.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 -- 172.21.15.52:0/2417389812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07880aa0c0 msgr2=0x7f07880ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 --2- 172.21.15.52:0/2417389812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07880aa0c0 0x7f07880ccb50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f07740097c0 tx=0x7f077401c430 comp rx=0 tx=0).stop 2024-05-04T12:18:51.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 -- 172.21.15.52:0/2417389812 shutdown_connections 2024-05-04T12:18:51.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 --2- 172.21.15.52:0/2417389812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07880aa0c0 0x7f07880ccb50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 -- 172.21.15.52:0/2417389812 >> 172.21.15.52:0/2417389812 conn(0x7f07880b14a0 msgr2=0x7f07880b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:51.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 -- 172.21.15.52:0/2417389812 shutdown_connections 2024-05-04T12:18:51.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 -- 172.21.15.52:0/2417389812 wait complete. 2024-05-04T12:18:51.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 Processor -- start 2024-05-04T12:18:51.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 -- start start 2024-05-04T12:18:51.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07880aa0c0 0x7f078812ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07880aa550 0x7f0788152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f078815bf50 0x7f0788144930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07880cd4a0 con 0x7f07880aa0c0 2024-05-04T12:18:51.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f078806cf70 con 0x7f07880aa550 2024-05-04T12:18:51.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078eace640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f078816a040 con 0x7f078815bf50 2024-05-04T12:18:51.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078c86a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07880aa0c0 0x7f078812ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078c86a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07880aa0c0 0x7f078812ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53740/0 (socket says 172.21.15.52:53740) 2024-05-04T12:18:51.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.361+0000 7f078c86a640 1 -- 172.21.15.52:0/3125860578 learned_addr learned my addr 172.21.15.52:0/3125860578 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:51.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f0787fff640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07880aa550 0x7f0788152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f078d06b640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f078815bf50 0x7f0788144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f078c86a640 1 -- 172.21.15.52:0/3125860578 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f078815bf50 msgr2=0x7f0788144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f078c86a640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f078815bf50 0x7f0788144930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f078c86a640 1 -- 172.21.15.52:0/3125860578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07880aa550 msgr2=0x7f0788152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f078c86a640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07880aa550 0x7f0788152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f078c86a640 1 -- 172.21.15.52:0/3125860578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0788154eb0 con 0x7f07880aa0c0 2024-05-04T12:18:51.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f0787fff640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07880aa550 0x7f0788152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:51.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f078c86a640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07880aa0c0 0x7f078812ffe0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f077c00b410 tx=0x7f077c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:51.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f07857fa640 1 -- 172.21.15.52:0/3125860578 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c015030 con 0x7f07880aa0c0 2024-05-04T12:18:51.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f07857fa640 1 -- 172.21.15.52:0/3125860578 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077c004020 con 0x7f07880aa0c0 2024-05-04T12:18:51.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f078eace640 1 -- 172.21.15.52:0/3125860578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0788155190 con 0x7f07880aa0c0 2024-05-04T12:18:51.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f07857fa640 1 -- 172.21.15.52:0/3125860578 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c004450 con 0x7f07880aa0c0 2024-05-04T12:18:51.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f078eace640 1 -- 172.21.15.52:0/3125860578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0788154b60 con 0x7f07880aa0c0 2024-05-04T12:18:51.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f078eace640 1 -- 172.21.15.52:0/3125860578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0788062e50 con 0x7f07880aa0c0 2024-05-04T12:18:51.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f07857fa640 1 -- 172.21.15.52:0/3125860578 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f077c018070 con 0x7f07880aa0c0 2024-05-04T12:18:51.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f07857fa640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f076c0678c0 0x7f076c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f07857fa640 1 -- 172.21.15.52:0/3125860578 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f077c0857b0 con 0x7f07880aa0c0 2024-05-04T12:18:51.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f0787fff640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f076c0678c0 0x7f076c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.365+0000 7f0787fff640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f076c0678c0 0x7f076c069d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f07740af000 tx=0x7f0774008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:51.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.369+0000 7f07857fa640 1 -- 172.21.15.52:0/3125860578 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f077c056380 con 0x7f07880aa0c0 2024-05-04T12:18:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.529+0000 7f078eace640 1 -- 172.21.15.52:0/3125860578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0788075320 con 0x7f07880aa0c0 2024-05-04T12:18:51.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.533+0000 7f07857fa640 1 -- 172.21.15.52:0/3125860578 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f077c055c80 con 0x7f07880aa0c0 2024-05-04T12:18:51.537 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:51.537 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:51.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.537+0000 7f078eace640 1 -- 172.21.15.52:0/3125860578 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f076c0678c0 msgr2=0x7f076c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.537+0000 7f078eace640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f076c0678c0 0x7f076c069d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f07740af000 tx=0x7f0774008040 comp rx=0 tx=0).stop 2024-05-04T12:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.537+0000 7f078eace640 1 -- 172.21.15.52:0/3125860578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07880aa0c0 msgr2=0x7f078812ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.537+0000 7f078eace640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07880aa0c0 0x7f078812ffe0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f077c00b410 tx=0x7f077c00b8d0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.537+0000 7f078eace640 1 -- 172.21.15.52:0/3125860578 shutdown_connections 2024-05-04T12:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.537+0000 7f078eace640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f076c0678c0 0x7f076c069d80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.537+0000 7f078eace640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f078815bf50 0x7f0788144930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.537+0000 7f078eace640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07880aa550 0x7f0788152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.537+0000 7f078eace640 1 --2- 172.21.15.52:0/3125860578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07880aa0c0 0x7f078812ffe0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.537+0000 7f078eace640 1 -- 172.21.15.52:0/3125860578 >> 172.21.15.52:0/3125860578 conn(0x7f07880b14a0 msgr2=0x7f078814a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.537+0000 7f078eace640 1 -- 172.21.15.52:0/3125860578 shutdown_connections 2024-05-04T12:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.537+0000 7f078eace640 1 -- 172.21.15.52:0/3125860578 wait complete. 2024-05-04T12:18:51.556 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:18:51.556 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:18:51.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2ca099640 1 Processor -- start 2024-05-04T12:18:51.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2ca099640 1 -- start start 2024-05-04T12:18:51.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2ca099640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff2c40cffc0 0x7ff2c415f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2ca099640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2c40b4660 0x7ff2c40aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2ca099640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c40b2f10 con 0x7ff2c40aefc0 2024-05-04T12:18:51.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2ca099640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c415f860 con 0x7ff2c40cffc0 2024-05-04T12:18:51.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2ca099640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c40b3180 con 0x7ff2c40b4660 2024-05-04T12:18:51.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2c37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff2c40cffc0 0x7ff2c415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2c2ffd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2c40b4660 0x7ff2c40aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2c3fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff2c40aefc0 0x7ff2c4161e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.52:43806/0 (socket says 172.21.15.52:43806) 2024-05-04T12:18:51.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2c37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff2c40cffc0 0x7ff2c415f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:47514/0 (socket says 172.21.15.52:47514) 2024-05-04T12:18:51.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2c3fff640 1 -- 172.21.15.52:0/3427136422 learned_addr learned my addr 172.21.15.52:0/3427136422 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:51.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2c37fe640 1 -- 172.21.15.52:0/3427136422 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2c40b4660 msgr2=0x7ff2c40aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:51.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2c37fe640 1 --2- 172.21.15.52:0/3427136422 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2c40b4660 0x7ff2c40aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2c37fe640 1 -- 172.21.15.52:0/3427136422 >> v1:172.21.15.111:6789/0 conn(0x7ff2c40aefc0 legacy=0x7ff2c4161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.677+0000 7ff2c37fe640 1 -- 172.21.15.52:0/3427136422 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c40b3460 con 0x7ff2c40cffc0 2024-05-04T12:18:51.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.681+0000 7ff2c37fe640 1 --2- 172.21.15.52:0/3427136422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c40cffc0 0x7ff2c415f320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40097c0 tx=0x7ff2b401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92927e7fd51672e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:51.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.681+0000 7ff2c27fc640 1 -- 172.21.15.52:0/3427136422 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40b2070 con 0x7ff2c40cffc0 2024-05-04T12:18:51.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.681+0000 7ff2c27fc640 1 -- 172.21.15.52:0/3427136422 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b40ad640 con 0x7ff2c40cffc0 2024-05-04T12:18:51.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.681+0000 7ff2c27fc640 1 -- 172.21.15.52:0/3427136422 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40ada10 con 0x7ff2c40cffc0 2024-05-04T12:18:51.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.681+0000 7ff2ca099640 1 -- 172.21.15.52:0/3427136422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c40cffc0 msgr2=0x7ff2c415f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.681+0000 7ff2ca099640 1 --2- 172.21.15.52:0/3427136422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c40cffc0 0x7ff2c415f320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40097c0 tx=0x7ff2b401c470 comp rx=0 tx=0).stop 2024-05-04T12:18:51.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.681+0000 7ff2ca099640 1 -- 172.21.15.52:0/3427136422 shutdown_connections 2024-05-04T12:18:51.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.681+0000 7ff2ca099640 1 --2- 172.21.15.52:0/3427136422 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2c40b4660 0x7ff2c40aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.681+0000 7ff2ca099640 1 --2- 172.21.15.52:0/3427136422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c40cffc0 0x7ff2c415f320 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.681+0000 7ff2ca099640 1 -- 172.21.15.52:0/3427136422 >> 172.21.15.52:0/3427136422 conn(0x7ff2c4071ac0 msgr2=0x7ff2c4071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:51.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.681+0000 7ff2ca099640 1 -- 172.21.15.52:0/3427136422 shutdown_connections 2024-05-04T12:18:51.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.681+0000 7ff2ca099640 1 -- 172.21.15.52:0/3427136422 wait complete. 2024-05-04T12:18:51.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.681+0000 7ff2ca099640 1 Processor -- start 2024-05-04T12:18:51.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.681+0000 7ff2ca099640 1 -- start start 2024-05-04T12:18:51.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2ca099640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2c40aefc0 0x7ff2c41584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2ca099640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2c40b4660 0x7ff2c41563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2ca099640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c40cffc0 0x7ff2c413cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2ca099640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c40af3d0 con 0x7ff2c40aefc0 2024-05-04T12:18:51.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2ca099640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c40cde20 con 0x7ff2c40cffc0 2024-05-04T12:18:51.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2ca099640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c40d03a0 con 0x7ff2c40b4660 2024-05-04T12:18:51.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2c37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2c40aefc0 0x7ff2c41584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2c37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2c40aefc0 0x7ff2c41584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53748/0 (socket says 172.21.15.52:53748) 2024-05-04T12:18:51.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2c37fe640 1 -- 172.21.15.52:0/37717398 learned_addr learned my addr 172.21.15.52:0/37717398 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:51.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2c3fff640 1 --2- 172.21.15.52:0/37717398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c40cffc0 0x7ff2c413cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2c2ffd640 1 --2- 172.21.15.52:0/37717398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2c40b4660 0x7ff2c41563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2c37fe640 1 -- 172.21.15.52:0/37717398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2c40b4660 msgr2=0x7ff2c41563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2c37fe640 1 --2- 172.21.15.52:0/37717398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2c40b4660 0x7ff2c41563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2c37fe640 1 -- 172.21.15.52:0/37717398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c40cffc0 msgr2=0x7ff2c413cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2c37fe640 1 --2- 172.21.15.52:0/37717398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c40cffc0 0x7ff2c413cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2c37fe640 1 -- 172.21.15.52:0/37717398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c415a850 con 0x7ff2c40aefc0 2024-05-04T12:18:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2c37fe640 1 --2- 172.21.15.52:0/37717398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2c40aefc0 0x7ff2c41584e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff2b401a0a0 tx=0x7ff2b4019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2abfff640 1 -- 172.21.15.52:0/37717398 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40ba020 con 0x7ff2c40aefc0 2024-05-04T12:18:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2abfff640 1 -- 172.21.15.52:0/37717398 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b40b6410 con 0x7ff2c40aefc0 2024-05-04T12:18:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2ca099640 1 -- 172.21.15.52:0/37717398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c4159200 con 0x7ff2c40aefc0 2024-05-04T12:18:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2abfff640 1 -- 172.21.15.52:0/37717398 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40b2040 con 0x7ff2c40aefc0 2024-05-04T12:18:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2ca099640 1 -- 172.21.15.52:0/37717398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c414f550 con 0x7ff2c40aefc0 2024-05-04T12:18:51.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2ca099640 1 -- 172.21.15.52:0/37717398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c414f790 con 0x7ff2c40aefc0 2024-05-04T12:18:51.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2abfff640 1 -- 172.21.15.52:0/37717398 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff2b40bd020 con 0x7ff2c40aefc0 2024-05-04T12:18:51.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2abfff640 1 --2- 172.21.15.52:0/37717398 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff2b00678c0 0x7ff2b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2abfff640 1 -- 172.21.15.52:0/37717398 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff2b411f9f0 con 0x7ff2c40aefc0 2024-05-04T12:18:51.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.685+0000 7ff2c2ffd640 1 --2- 172.21.15.52:0/37717398 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff2b00678c0 0x7ff2b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.689+0000 7ff2c2ffd640 1 --2- 172.21.15.52:0/37717398 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff2b00678c0 0x7ff2b0069d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4158260 tx=0x7ff2b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:51.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.689+0000 7ff2abfff640 1 -- 172.21.15.52:0/37717398 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff2c414f790 con 0x7ff2c40aefc0 2024-05-04T12:18:51.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.853+0000 7ff2ca099640 1 -- 172.21.15.52:0/37717398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff2c4075320 con 0x7ff2c40aefc0 2024-05-04T12:18:51.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.853+0000 7ff2abfff640 1 -- 172.21.15.52:0/37717398 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7ff2b40c4030 con 0x7ff2c40aefc0 2024-05-04T12:18:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:18:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.857+0000 7ff2ca099640 1 -- 172.21.15.52:0/37717398 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff2b00678c0 msgr2=0x7ff2b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.857+0000 7ff2ca099640 1 --2- 172.21.15.52:0/37717398 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff2b00678c0 0x7ff2b0069d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4158260 tx=0x7ff2b8008040 comp rx=0 tx=0).stop 2024-05-04T12:18:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.857+0000 7ff2ca099640 1 -- 172.21.15.52:0/37717398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2c40aefc0 msgr2=0x7ff2c41584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.857+0000 7ff2ca099640 1 --2- 172.21.15.52:0/37717398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2c40aefc0 0x7ff2c41584e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff2b401a0a0 tx=0x7ff2b4019210 comp rx=0 tx=0).stop 2024-05-04T12:18:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.857+0000 7ff2ca099640 1 -- 172.21.15.52:0/37717398 shutdown_connections 2024-05-04T12:18:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.857+0000 7ff2ca099640 1 --2- 172.21.15.52:0/37717398 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff2b00678c0 0x7ff2b0069d80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.857+0000 7ff2ca099640 1 --2- 172.21.15.52:0/37717398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c40cffc0 0x7ff2c413cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.857+0000 7ff2ca099640 1 --2- 172.21.15.52:0/37717398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2c40b4660 0x7ff2c41563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.857+0000 7ff2ca099640 1 --2- 172.21.15.52:0/37717398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2c40aefc0 0x7ff2c41584e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.857+0000 7ff2ca099640 1 -- 172.21.15.52:0/37717398 >> 172.21.15.52:0/37717398 conn(0x7ff2c4071ac0 msgr2=0x7ff2c4155810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.857+0000 7ff2ca099640 1 -- 172.21.15.52:0/37717398 shutdown_connections 2024-05-04T12:18:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.857+0000 7ff2ca099640 1 -- 172.21.15.52:0/37717398 wait complete. 2024-05-04T12:18:51.876 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:18:51.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f7738e9c640 1 Processor -- start 2024-05-04T12:18:51.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f7738e9c640 1 -- start start 2024-05-04T12:18:51.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f7738e9c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7734070b60 0x7f77341601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f7738e9c640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77340c5850 con 0x7f77340706d0 2024-05-04T12:18:51.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f7738e9c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77340d0050 con 0x7f7734159a60 2024-05-04T12:18:51.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f7738e9c640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f773415c4a0 con 0x7f7734070b60 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f7731d9b640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f77340706d0 0x7f773415fae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55140/0 (socket says 172.21.15.52:55140) 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f7731d9b640 1 -- 172.21.15.52:0/2362387541 learned_addr learned my addr 172.21.15.52:0/2362387541 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f7732d9d640 1 --2- 172.21.15.52:0/2362387541 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7734070b60 0x7f77341601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 -- 172.21.15.52:0/2362387541 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2459056971 0 0) 0x7f77340c5850 con 0x7f77340706d0 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 -- 172.21.15.52:0/2362387541 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f771c003530 con 0x7f77340706d0 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 -- 172.21.15.52:0/2362387541 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3156902102 0 0) 0x7f771c003530 con 0x7f77340706d0 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 -- 172.21.15.52:0/2362387541 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7734070b60 msgr2=0x7f77341601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 --2- 172.21.15.52:0/2362387541 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7734070b60 0x7f77341601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 -- 172.21.15.52:0/2362387541 >> v1:172.21.15.111:6789/0 conn(0x7f7734159a60 legacy=0x7f77340c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 -- 172.21.15.52:0/2362387541 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77340d0330 con 0x7f77340706d0 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 -- 172.21.15.52:0/2362387541 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f7728001680 con 0x7f77340706d0 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 -- 172.21.15.52:0/2362387541 >> v1:172.21.15.52:6789/0 conn(0x7f77340706d0 legacy=0x7f773415fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 --2- 172.21.15.52:0/2362387541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f771c0039f0 0x7f771c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 --2- 172.21.15.52:0/2362387541 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7734070b60 0x7f771c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:51.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 --2- 172.21.15.52:0/2362387541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f771c004d50 0x7f771c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 -- 172.21.15.52:0/2362387541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f773415c4a0 con 0x7f771c0039f0 2024-05-04T12:18:52.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 -- 172.21.15.52:0/2362387541 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f771c00bb90 con 0x7f771c004d50 2024-05-04T12:18:52.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773159a640 1 -- 172.21.15.52:0/2362387541 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f771c00bd90 con 0x7f7734070b60 2024-05-04T12:18:52.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773259c640 1 --2- 172.21.15.52:0/2362387541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f771c0039f0 0x7f771c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f7732d9d640 1 --2- 172.21.15.52:0/2362387541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f771c004d50 0x7f771c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773259c640 1 -- 172.21.15.52:0/2362387541 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7734070b60 msgr2=0x7f771c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:52.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773259c640 1 --2- 172.21.15.52:0/2362387541 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7734070b60 0x7f771c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773259c640 1 -- 172.21.15.52:0/2362387541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f771c004d50 msgr2=0x7f771c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:52.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773259c640 1 --2- 172.21.15.52:0/2362387541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f771c004d50 0x7f771c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.993+0000 7f773259c640 1 -- 172.21.15.52:0/2362387541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771c003530 con 0x7f771c0039f0 2024-05-04T12:18:52.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.997+0000 7f773259c640 1 --2- 172.21.15.52:0/2362387541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f771c0039f0 0x7f771c003e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7724001780 tx=0x7f772401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a605168e982ad3f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:52.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.997+0000 7f773159a640 1 -- 172.21.15.52:0/2362387541 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77240013a0 con 0x7f771c0039f0 2024-05-04T12:18:52.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.997+0000 7f773159a640 1 -- 172.21.15.52:0/2362387541 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77240052f0 con 0x7f771c0039f0 2024-05-04T12:18:52.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.997+0000 7f773159a640 1 -- 172.21.15.52:0/2362387541 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724003840 con 0x7f771c0039f0 2024-05-04T12:18:52.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.997+0000 7f7738e9c640 1 -- 172.21.15.52:0/2362387541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f771c0039f0 msgr2=0x7f771c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.997+0000 7f7738e9c640 1 --2- 172.21.15.52:0/2362387541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f771c0039f0 0x7f771c003e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7724001780 tx=0x7f772401c420 comp rx=0 tx=0).stop 2024-05-04T12:18:52.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.997+0000 7f7732d9d640 1 -- 172.21.15.52:0/2362387541 reap_dead start 2024-05-04T12:18:52.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.997+0000 7f7738e9c640 1 -- 172.21.15.52:0/2362387541 shutdown_connections 2024-05-04T12:18:52.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.997+0000 7f7738e9c640 1 -- 172.21.15.52:0/2362387541 >> 172.21.15.52:0/2362387541 conn(0x7f77340d12c0 msgr2=0x7f77340d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:52.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.997+0000 7f7738e9c640 1 -- 172.21.15.52:0/2362387541 shutdown_connections 2024-05-04T12:18:52.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.997+0000 7f7738e9c640 1 -- 172.21.15.52:0/2362387541 wait complete. 2024-05-04T12:18:52.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.997+0000 7f7738e9c640 1 Processor -- start 2024-05-04T12:18:52.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:51.997+0000 7f7738e9c640 1 -- start start 2024-05-04T12:18:52.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7738e9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f771c0039f0 0x7f773414dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7738e9c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7734145680 0x7f773414f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7738e9c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77340c71e0 0x7f77340c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7738e9c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f77340d0c90 con 0x7f77340c71e0 2024-05-04T12:18:52.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7738e9c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f77340d0b10 con 0x7f771c0039f0 2024-05-04T12:18:52.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7738e9c640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f77340c54a0 con 0x7f7734145680 2024-05-04T12:18:52.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7732d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77340c71e0 0x7f77340c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7732d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77340c71e0 0x7f77340c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53758/0 (socket says 172.21.15.52:53758) 2024-05-04T12:18:52.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7732d9d640 1 -- 172.21.15.52:0/3244384049 learned_addr learned my addr 172.21.15.52:0/3244384049 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:52.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f773259c640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f771c0039f0 0x7f773414dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7731d9b640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7734145680 0x7f773414f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7732d9d640 1 -- 172.21.15.52:0/3244384049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7734145680 msgr2=0x7f773414f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7732d9d640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7734145680 0x7f773414f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7732d9d640 1 -- 172.21.15.52:0/3244384049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f771c0039f0 msgr2=0x7f773414dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7732d9d640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f771c0039f0 0x7f773414dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7732d9d640 1 -- 172.21.15.52:0/3244384049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773414e5d0 con 0x7f77340c71e0 2024-05-04T12:18:52.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7731d9b640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7734145680 0x7f773414f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:52.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f773259c640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f771c0039f0 0x7f773414dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:52.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7732d9d640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77340c71e0 0x7f77340c7660 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7734071760 tx=0x7f772c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:52.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7722ffd640 1 -- 172.21.15.52:0/3244384049 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c00fe30 con 0x7f77340c71e0 2024-05-04T12:18:52.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7722ffd640 1 -- 172.21.15.52:0/3244384049 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f772c0039a0 con 0x7f77340c71e0 2024-05-04T12:18:52.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7738e9c640 1 -- 172.21.15.52:0/3244384049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773414e8b0 con 0x7f77340c71e0 2024-05-04T12:18:52.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7722ffd640 1 -- 172.21.15.52:0/3244384049 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c003d90 con 0x7f77340c71e0 2024-05-04T12:18:52.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7738e9c640 1 -- 172.21.15.52:0/3244384049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773414b970 con 0x7f77340c71e0 2024-05-04T12:18:52.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7738e9c640 1 -- 172.21.15.52:0/3244384049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f773414bb70 con 0x7f77340c71e0 2024-05-04T12:18:52.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7722ffd640 1 -- 172.21.15.52:0/3244384049 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f772c003f30 con 0x7f77340c71e0 2024-05-04T12:18:52.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7722ffd640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f77140678f0 0x7f7714069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f7722ffd640 1 -- 172.21.15.52:0/3244384049 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f772c086140 con 0x7f77340c71e0 2024-05-04T12:18:52.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f773259c640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f77140678f0 0x7f7714069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.001+0000 7f773259c640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f77140678f0 0x7f7714069db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7724001750 tx=0x7f77240011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:52.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.005+0000 7f7722ffd640 1 -- 172.21.15.52:0/3244384049 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f773414bb70 con 0x7f77340c71e0 2024-05-04T12:18:52.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.173+0000 7f7738e9c640 1 -- 172.21.15.52:0/3244384049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7734062e50 con 0x7f77340c71e0 2024-05-04T12:18:52.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.173+0000 7f7722ffd640 1 -- 172.21.15.52:0/3244384049 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f772c012070 con 0x7f77340c71e0 2024-05-04T12:18:52.176 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:18:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.173+0000 7f7738e9c640 1 -- 172.21.15.52:0/3244384049 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f77140678f0 msgr2=0x7f7714069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.173+0000 7f7738e9c640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f77140678f0 0x7f7714069db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7724001750 tx=0x7f77240011a0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.173+0000 7f7738e9c640 1 -- 172.21.15.52:0/3244384049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77340c71e0 msgr2=0x7f77340c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.173+0000 7f7738e9c640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77340c71e0 0x7f77340c7660 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7734071760 tx=0x7f772c00f4a0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.173+0000 7f7738e9c640 1 -- 172.21.15.52:0/3244384049 shutdown_connections 2024-05-04T12:18:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.173+0000 7f7738e9c640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f77140678f0 0x7f7714069db0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.173+0000 7f7738e9c640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77340c71e0 0x7f77340c7660 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.173+0000 7f7738e9c640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7734145680 0x7f773414f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.173+0000 7f7738e9c640 1 --2- 172.21.15.52:0/3244384049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f771c0039f0 0x7f773414dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.173+0000 7f7738e9c640 1 -- 172.21.15.52:0/3244384049 >> 172.21.15.52:0/3244384049 conn(0x7f77340d12c0 msgr2=0x7f773415b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.177+0000 7f7738e9c640 1 -- 172.21.15.52:0/3244384049 shutdown_connections 2024-05-04T12:18:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.177+0000 7f7738e9c640 1 -- 172.21.15.52:0/3244384049 wait complete. 2024-05-04T12:18:52.194 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:18:52.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.309+0000 7f3375e74640 1 Processor -- start 2024-05-04T12:18:52.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.309+0000 7f3375e74640 1 -- start start 2024-05-04T12:18:52.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.309+0000 7f3375e74640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f33700ccea0 0x7f3370166f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.309+0000 7f3375e74640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f337006dbf0 con 0x7f337015c590 2024-05-04T12:18:52.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.309+0000 7f3375e74640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f337006f090 con 0x7f3370145960 2024-05-04T12:18:52.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.309+0000 7f3375e74640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f337006cfe0 con 0x7f33700ccea0 2024-05-04T12:18:52.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.309+0000 7f336effd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f337015c590 0x7f33700cc790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55152/0 (socket says 172.21.15.52:55152) 2024-05-04T12:18:52.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.309+0000 7f336effd640 1 -- 172.21.15.52:0/3666961734 learned_addr learned my addr 172.21.15.52:0/3666961734 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:52.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.309+0000 7f336ffff640 1 --2- 172.21.15.52:0/3666961734 >> v2:172.21.15.52:3300/0 conn(0x7f33700ccea0 0x7f3370166f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.309+0000 7f336e7fc640 1 -- 172.21.15.52:0/3666961734 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1778332739 0 0) 0x7f337006dbf0 con 0x7f337015c590 2024-05-04T12:18:52.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.309+0000 7f336e7fc640 1 -- 172.21.15.52:0/3666961734 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3358003530 con 0x7f337015c590 2024-05-04T12:18:52.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.309+0000 7f336ffff640 1 -- 172.21.15.52:0/3666961734 >> v1:172.21.15.111:6789/0 conn(0x7f3370145960 legacy=0x7f3370160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.309+0000 7f336ffff640 1 -- 172.21.15.52:0/3666961734 >> v1:172.21.15.52:6789/0 conn(0x7f337015c590 legacy=0x7f33700cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.309+0000 7f336ffff640 1 -- 172.21.15.52:0/3666961734 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337006d2c0 con 0x7f33700ccea0 2024-05-04T12:18:52.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f336ffff640 1 --2- 172.21.15.52:0/3666961734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33700ccea0 0x7f3370166f10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3368009d90 tx=0x7f336801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbf05e9cd3fcc0a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:52.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f336e7fc640 1 -- 172.21.15.52:0/3666961734 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33680b2070 con 0x7f33700ccea0 2024-05-04T12:18:52.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f336e7fc640 1 -- 172.21.15.52:0/3666961734 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33680a5970 con 0x7f33700ccea0 2024-05-04T12:18:52.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f336e7fc640 1 -- 172.21.15.52:0/3666961734 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33680a5d20 con 0x7f33700ccea0 2024-05-04T12:18:52.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 -- 172.21.15.52:0/3666961734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33700ccea0 msgr2=0x7f3370166f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 --2- 172.21.15.52:0/3666961734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33700ccea0 0x7f3370166f10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3368009d90 tx=0x7f336801c8b0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 -- 172.21.15.52:0/3666961734 shutdown_connections 2024-05-04T12:18:52.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 --2- 172.21.15.52:0/3666961734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33700ccea0 0x7f3370166f10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 -- 172.21.15.52:0/3666961734 >> 172.21.15.52:0/3666961734 conn(0x7f3370067730 msgr2=0x7f33700b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:52.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 -- 172.21.15.52:0/3666961734 shutdown_connections 2024-05-04T12:18:52.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 -- 172.21.15.52:0/3666961734 wait complete. 2024-05-04T12:18:52.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 Processor -- start 2024-05-04T12:18:52.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 -- start start 2024-05-04T12:18:52.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33700ccea0 0x7f337015b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3370145960 0x7f33701fa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f337015c590 0x7f337015d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f337006cdc0 con 0x7f33700ccea0 2024-05-04T12:18:52.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f337006cc40 con 0x7f3370145960 2024-05-04T12:18:52.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f3375e74640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f3370145d40 con 0x7f337015c590 2024-05-04T12:18:52.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f336f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33700ccea0 0x7f337015b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f336f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33700ccea0 0x7f337015b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53770/0 (socket says 172.21.15.52:53770) 2024-05-04T12:18:52.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.313+0000 7f336f7fe640 1 -- 172.21.15.52:0/1844426694 learned_addr learned my addr 172.21.15.52:0/1844426694 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:52.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f336ffff640 1 --2- 172.21.15.52:0/1844426694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f337015c590 0x7f337015d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f336effd640 1 --2- 172.21.15.52:0/1844426694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3370145960 0x7f33701fa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f336f7fe640 1 -- 172.21.15.52:0/1844426694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f337015c590 msgr2=0x7f337015d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f336f7fe640 1 --2- 172.21.15.52:0/1844426694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f337015c590 0x7f337015d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f336f7fe640 1 -- 172.21.15.52:0/1844426694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3370145960 msgr2=0x7f33701fa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f336f7fe640 1 --2- 172.21.15.52:0/1844426694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3370145960 0x7f33701fa7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f336f7fe640 1 -- 172.21.15.52:0/1844426694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33701fb140 con 0x7f33700ccea0 2024-05-04T12:18:52.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f336f7fe640 1 --2- 172.21.15.52:0/1844426694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33700ccea0 0x7f337015b620 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f336000c530 tx=0x7f336000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:52.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f3357fff640 1 -- 172.21.15.52:0/1844426694 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360016020 con 0x7f33700ccea0 2024-05-04T12:18:52.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f3357fff640 1 -- 172.21.15.52:0/1844426694 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3360005220 con 0x7f33700ccea0 2024-05-04T12:18:52.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f3375e74640 1 -- 172.21.15.52:0/1844426694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33701fb420 con 0x7f33700ccea0 2024-05-04T12:18:52.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f3357fff640 1 -- 172.21.15.52:0/1844426694 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360007030 con 0x7f33700ccea0 2024-05-04T12:18:52.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f3375e74640 1 -- 172.21.15.52:0/1844426694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f337015bb60 con 0x7f33700ccea0 2024-05-04T12:18:52.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f3375e74640 1 -- 172.21.15.52:0/1844426694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3370062e50 con 0x7f33700ccea0 2024-05-04T12:18:52.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f3357fff640 1 -- 172.21.15.52:0/1844426694 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f33600071d0 con 0x7f33700ccea0 2024-05-04T12:18:52.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f3357fff640 1 --2- 172.21.15.52:0/1844426694 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f335c0678c0 0x7f335c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f3357fff640 1 -- 172.21.15.52:0/1844426694 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f33600856b0 con 0x7f33700ccea0 2024-05-04T12:18:52.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f336effd640 1 --2- 172.21.15.52:0/1844426694 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f335c0678c0 0x7f335c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.317+0000 7f336effd640 1 --2- 172.21.15.52:0/1844426694 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f335c0678c0 0x7f335c069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3370166630 tx=0x7f3364009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:52.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.321+0000 7f3357fff640 1 -- 172.21.15.52:0/1844426694 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3360020020 con 0x7f33700ccea0 2024-05-04T12:18:52.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.473+0000 7f3375e74640 1 -- 172.21.15.52:0/1844426694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3370075320 con 0x7f33700ccea0 2024-05-04T12:18:52.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.477+0000 7f3357fff640 1 -- 172.21.15.52:0/1844426694 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f33600074c0 con 0x7f33700ccea0 2024-05-04T12:18:52.479 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:18:52.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.477+0000 7f3375e74640 1 -- 172.21.15.52:0/1844426694 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f335c0678c0 msgr2=0x7f335c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.477+0000 7f3375e74640 1 --2- 172.21.15.52:0/1844426694 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f335c0678c0 0x7f335c069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3370166630 tx=0x7f3364009040 comp rx=0 tx=0).stop 2024-05-04T12:18:52.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.477+0000 7f3375e74640 1 -- 172.21.15.52:0/1844426694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33700ccea0 msgr2=0x7f337015b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.477+0000 7f3375e74640 1 --2- 172.21.15.52:0/1844426694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33700ccea0 0x7f337015b620 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f336000c530 tx=0x7f336000c9f0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.477+0000 7f3375e74640 1 -- 172.21.15.52:0/1844426694 shutdown_connections 2024-05-04T12:18:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.477+0000 7f3375e74640 1 --2- 172.21.15.52:0/1844426694 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f335c0678c0 0x7f335c069d80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.477+0000 7f3375e74640 1 --2- 172.21.15.52:0/1844426694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f337015c590 0x7f337015d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.477+0000 7f3375e74640 1 --2- 172.21.15.52:0/1844426694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3370145960 0x7f33701fa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.477+0000 7f3375e74640 1 --2- 172.21.15.52:0/1844426694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33700ccea0 0x7f337015b620 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.477+0000 7f3375e74640 1 -- 172.21.15.52:0/1844426694 >> 172.21.15.52:0/1844426694 conn(0x7f3370067730 msgr2=0x7f337013d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.477+0000 7f3375e74640 1 -- 172.21.15.52:0/1844426694 shutdown_connections 2024-05-04T12:18:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.477+0000 7f3375e74640 1 -- 172.21.15.52:0/1844426694 wait complete. 2024-05-04T12:18:52.505 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-05-04T12:18:52.506 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f945a866640 1 Processor -- start 2024-05-04T12:18:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f945a866640 1 -- start start 2024-05-04T12:18:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f945a866640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f945415c7f0 0x7f94540723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f945a866640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9454072aa0 con 0x7f94540ccdb0 2024-05-04T12:18:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f945a866640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94540708e0 con 0x7f94540cd190 2024-05-04T12:18:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f945a866640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f9454070690 con 0x7f945415c7f0 2024-05-04T12:18:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f9453fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f94540ccdb0 0x7f9454158640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55168/0 (socket says 172.21.15.52:55168) 2024-05-04T12:18:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f9453fff640 1 -- 172.21.15.52:0/1370780469 learned_addr learned my addr 172.21.15.52:0/1370780469 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f944bfff640 1 --2- 172.21.15.52:0/1370780469 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f945415c7f0 0x7f94540723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f94537fe640 1 -- 172.21.15.52:0/1370780469 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2946802272 0 0) 0x7f9454072aa0 con 0x7f94540ccdb0 2024-05-04T12:18:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f94537fe640 1 -- 172.21.15.52:0/1370780469 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9440003530 con 0x7f94540ccdb0 2024-05-04T12:18:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f944bfff640 1 -- 172.21.15.52:0/1370780469 >> v1:172.21.15.111:6789/0 conn(0x7f94540cd190 legacy=0x7f945406de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f94537fe640 1 -- 172.21.15.52:0/1370780469 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2327409498 0 0) 0x7f9440003530 con 0x7f94540ccdb0 2024-05-04T12:18:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f944bfff640 1 -- 172.21.15.52:0/1370780469 >> v1:172.21.15.52:6789/0 conn(0x7f94540ccdb0 legacy=0x7f9454158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.625+0000 7f944bfff640 1 -- 172.21.15.52:0/1370780469 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9454070bc0 con 0x7f945415c7f0 2024-05-04T12:18:52.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f944bfff640 1 --2- 172.21.15.52:0/1370780469 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f945415c7f0 0x7f94540723e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f943c0097f0 tx=0x7f943c01c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1cd67206127e268 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:52.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f94537fe640 1 -- 172.21.15.52:0/1370780469 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c0b0070 con 0x7f945415c7f0 2024-05-04T12:18:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f94537fe640 1 -- 172.21.15.52:0/1370780469 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f943c0ab400 con 0x7f945415c7f0 2024-05-04T12:18:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f94537fe640 1 -- 172.21.15.52:0/1370780469 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444001680 con 0x7f945415c7f0 2024-05-04T12:18:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 -- 172.21.15.52:0/1370780469 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f945415c7f0 msgr2=0x7f94540723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 --2- 172.21.15.52:0/1370780469 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f945415c7f0 0x7f94540723e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f943c0097f0 tx=0x7f943c01c370 comp rx=0 tx=0).stop 2024-05-04T12:18:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 -- 172.21.15.52:0/1370780469 shutdown_connections 2024-05-04T12:18:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 --2- 172.21.15.52:0/1370780469 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f945415c7f0 0x7f94540723e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 -- 172.21.15.52:0/1370780469 >> 172.21.15.52:0/1370780469 conn(0x7f94540b14a0 msgr2=0x7f94540b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 -- 172.21.15.52:0/1370780469 shutdown_connections 2024-05-04T12:18:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 -- 172.21.15.52:0/1370780469 wait complete. 2024-05-04T12:18:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 Processor -- start 2024-05-04T12:18:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 -- start start 2024-05-04T12:18:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94540ccdb0 0x7f945415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94540cd190 0x7f94541518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f945415c7f0 0x7f9454151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9454169010 con 0x7f945415c7f0 2024-05-04T12:18:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9454168e90 con 0x7f94540ccdb0 2024-05-04T12:18:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.629+0000 7f945a866640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f9454169190 con 0x7f94540cd190 2024-05-04T12:18:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f9458e03640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f945415c7f0 0x7f9454151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f9458e03640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f945415c7f0 0x7f9454151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53782/0 (socket says 172.21.15.52:53782) 2024-05-04T12:18:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f9458e03640 1 -- 172.21.15.52:0/915242749 learned_addr learned my addr 172.21.15.52:0/915242749 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f944bfff640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94540cd190 0x7f94541518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f9453fff640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94540ccdb0 0x7f945415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f9458e03640 1 -- 172.21.15.52:0/915242749 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94540cd190 msgr2=0x7f94541518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f9458e03640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94540cd190 0x7f94541518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f9458e03640 1 -- 172.21.15.52:0/915242749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94540ccdb0 msgr2=0x7f945415ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f9458e03640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94540ccdb0 0x7f945415ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f9458e03640 1 -- 172.21.15.52:0/915242749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9454150470 con 0x7f945415c7f0 2024-05-04T12:18:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f944bfff640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94540cd190 0x7f94541518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f9458e03640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f945415c7f0 0x7f9454151e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f944c00c730 tx=0x7f944c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f94517fa640 1 -- 172.21.15.52:0/915242749 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c003b80 con 0x7f945415c7f0 2024-05-04T12:18:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f94517fa640 1 -- 172.21.15.52:0/915242749 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944c003d20 con 0x7f945415c7f0 2024-05-04T12:18:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f945a866640 1 -- 172.21.15.52:0/915242749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94540ad250 con 0x7f945415c7f0 2024-05-04T12:18:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f94517fa640 1 -- 172.21.15.52:0/915242749 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c00aa30 con 0x7f945415c7f0 2024-05-04T12:18:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f945a866640 1 -- 172.21.15.52:0/915242749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9454156f30 con 0x7f945415c7f0 2024-05-04T12:18:52.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f945a866640 1 -- 172.21.15.52:0/915242749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9454062e50 con 0x7f945415c7f0 2024-05-04T12:18:52.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f94517fa640 1 -- 172.21.15.52:0/915242749 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f944c00abd0 con 0x7f945415c7f0 2024-05-04T12:18:52.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f94517fa640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f94340678c0 0x7f9434069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f94517fa640 1 -- 172.21.15.52:0/915242749 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f944c085b40 con 0x7f945415c7f0 2024-05-04T12:18:52.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f9453fff640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f94340678c0 0x7f9434069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.633+0000 7f9453fff640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f94340678c0 0x7f9434069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9444005890 tx=0x7f9444009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:52.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.637+0000 7f94517fa640 1 -- 172.21.15.52:0/915242749 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9454150470 con 0x7f945415c7f0 2024-05-04T12:18:52.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.793+0000 7f945a866640 1 -- 172.21.15.52:0/915242749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9454155f80 con 0x7f945415c7f0 2024-05-04T12:18:52.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.797+0000 7f94517fa640 1 -- 172.21.15.52:0/915242749 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f944c056680 con 0x7f945415c7f0 2024-05-04T12:18:52.801 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:52.801 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:52.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.801+0000 7f945a866640 1 -- 172.21.15.52:0/915242749 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f94340678c0 msgr2=0x7f9434069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.801+0000 7f945a866640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f94340678c0 0x7f9434069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9444005890 tx=0x7f9444009040 comp rx=0 tx=0).stop 2024-05-04T12:18:52.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.801+0000 7f945a866640 1 -- 172.21.15.52:0/915242749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f945415c7f0 msgr2=0x7f9454151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.801+0000 7f945a866640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f945415c7f0 0x7f9454151e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f944c00c730 tx=0x7f944c00cbf0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.801+0000 7f945a866640 1 -- 172.21.15.52:0/915242749 shutdown_connections 2024-05-04T12:18:52.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.801+0000 7f945a866640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f94340678c0 0x7f9434069d80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.801+0000 7f945a866640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f945415c7f0 0x7f9454151e10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.801+0000 7f945a866640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94540cd190 0x7f94541518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.801+0000 7f945a866640 1 --2- 172.21.15.52:0/915242749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94540ccdb0 0x7f945415ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.801+0000 7f945a866640 1 -- 172.21.15.52:0/915242749 >> 172.21.15.52:0/915242749 conn(0x7f94540b14a0 msgr2=0x7f945415d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:52.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.801+0000 7f945a866640 1 -- 172.21.15.52:0/915242749 shutdown_connections 2024-05-04T12:18:52.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.801+0000 7f945a866640 1 -- 172.21.15.52:0/915242749 wait complete. 2024-05-04T12:18:52.820 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:18:52.820 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:18:52.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d32d6640 1 Processor -- start 2024-05-04T12:18:52.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d32d6640 1 -- start start 2024-05-04T12:18:52.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d32d6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb0cc15e180 0x7fb0cc0ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d32d6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb0cc162660 0x7fb0cc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d32d6640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0cc0ce6a0 con 0x7fb0cc070c10 2024-05-04T12:18:52.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d32d6640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb0cc0cf220 con 0x7fb0cc15e180 2024-05-04T12:18:52.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d32d6640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fb0cc0ce980 con 0x7fb0cc162660 2024-05-04T12:18:52.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d1072640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb0cc15e180 0x7fb0cc0ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d1072640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb0cc15e180 0x7fb0cc0ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53798/0 (socket says 172.21.15.52:53798) 2024-05-04T12:18:52.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d1072640 1 -- 172.21.15.52:0/3467454741 learned_addr learned my addr 172.21.15.52:0/3467454741 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:52.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d0871640 1 --2- 172.21.15.52:0/3467454741 >> v2:172.21.15.111:3300/0 conn(0x7fb0cc162660 0x7fb0cc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d1072640 1 -- 172.21.15.52:0/3467454741 >> v2:172.21.15.111:3300/0 conn(0x7fb0cc162660 msgr2=0x7fb0cc0706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:52.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d1072640 1 --2- 172.21.15.52:0/3467454741 >> v2:172.21.15.111:3300/0 conn(0x7fb0cc162660 0x7fb0cc0706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d1072640 1 -- 172.21.15.52:0/3467454741 >> v1:172.21.15.111:6789/0 conn(0x7fb0cc070c10 legacy=0x7fb0cc06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:52.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.941+0000 7fb0d1072640 1 -- 172.21.15.52:0/3467454741 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0cc0cec60 con 0x7fb0cc15e180 2024-05-04T12:18:52.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.945+0000 7fb0d1072640 1 --2- 172.21.15.52:0/3467454741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0cc15e180 0x7fb0cc0ce0c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc071430 tx=0x7fb0c401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a782d56a5f30a24 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:52.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.945+0000 7fb0c3fff640 1 -- 172.21.15.52:0/3467454741 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40a56c0 con 0x7fb0cc15e180 2024-05-04T12:18:52.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.945+0000 7fb0c3fff640 1 -- 172.21.15.52:0/3467454741 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c40a5860 con 0x7fb0cc15e180 2024-05-04T12:18:52.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.945+0000 7fb0c3fff640 1 -- 172.21.15.52:0/3467454741 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40a5c30 con 0x7fb0cc15e180 2024-05-04T12:18:52.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.945+0000 7fb0d32d6640 1 -- 172.21.15.52:0/3467454741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0cc15e180 msgr2=0x7fb0cc0ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.945+0000 7fb0d32d6640 1 --2- 172.21.15.52:0/3467454741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0cc15e180 0x7fb0cc0ce0c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc071430 tx=0x7fb0c401c9f0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.945+0000 7fb0d32d6640 1 -- 172.21.15.52:0/3467454741 shutdown_connections 2024-05-04T12:18:52.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.945+0000 7fb0d32d6640 1 --2- 172.21.15.52:0/3467454741 >> v2:172.21.15.111:3300/0 conn(0x7fb0cc162660 0x7fb0cc0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.945+0000 7fb0d32d6640 1 --2- 172.21.15.52:0/3467454741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0cc15e180 0x7fb0cc0ce0c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.945+0000 7fb0d32d6640 1 -- 172.21.15.52:0/3467454741 >> 172.21.15.52:0/3467454741 conn(0x7fb0cc0d12c0 msgr2=0x7fb0cc0d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:52.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.945+0000 7fb0d32d6640 1 -- 172.21.15.52:0/3467454741 shutdown_connections 2024-05-04T12:18:52.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.945+0000 7fb0d32d6640 1 -- 172.21.15.52:0/3467454741 wait complete. 2024-05-04T12:18:52.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.945+0000 7fb0d32d6640 1 Processor -- start 2024-05-04T12:18:52.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.945+0000 7fb0d32d6640 1 -- start start 2024-05-04T12:18:52.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d32d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0cc070c10 0x7fb0cc1478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d32d6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0cc15e180 0x7fb0cc160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d32d6640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0cc162660 0x7fb0cc071d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d32d6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc16b170 con 0x7fb0cc070c10 2024-05-04T12:18:52.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d32d6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc16aff0 con 0x7fb0cc15e180 2024-05-04T12:18:52.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d32d6640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fb0cc16b2f0 con 0x7fb0cc162660 2024-05-04T12:18:52.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d1072640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0cc070c10 0x7fb0cc1478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d1072640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0cc070c10 0x7fb0cc1478d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53802/0 (socket says 172.21.15.52:53802) 2024-05-04T12:18:52.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d1072640 1 -- 172.21.15.52:0/2703230611 learned_addr learned my addr 172.21.15.52:0/2703230611 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:52.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d0871640 1 --2- 172.21.15.52:0/2703230611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0cc15e180 0x7fb0cc160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d1873640 1 --2- 172.21.15.52:0/2703230611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0cc162660 0x7fb0cc071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d1072640 1 -- 172.21.15.52:0/2703230611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0cc162660 msgr2=0x7fb0cc071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d1072640 1 --2- 172.21.15.52:0/2703230611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0cc162660 0x7fb0cc071d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d1072640 1 -- 172.21.15.52:0/2703230611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0cc15e180 msgr2=0x7fb0cc160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:52.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d1072640 1 --2- 172.21.15.52:0/2703230611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0cc15e180 0x7fb0cc160760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:52.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d1072640 1 -- 172.21.15.52:0/2703230611 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0cc161930 con 0x7fb0cc070c10 2024-05-04T12:18:52.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d1072640 1 --2- 172.21.15.52:0/2703230611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0cc070c10 0x7fb0cc1478d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40af040 tx=0x7fb0c4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:52.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0c1ffb640 1 -- 172.21.15.52:0/2703230611 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40bc020 con 0x7fb0cc070c10 2024-05-04T12:18:52.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0c1ffb640 1 -- 172.21.15.52:0/2703230611 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c40b8690 con 0x7fb0cc070c10 2024-05-04T12:18:52.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d32d6640 1 -- 172.21.15.52:0/2703230611 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0cc15eab0 con 0x7fb0cc070c10 2024-05-04T12:18:52.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0c1ffb640 1 -- 172.21.15.52:0/2703230611 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40b8a80 con 0x7fb0cc070c10 2024-05-04T12:18:52.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d32d6640 1 -- 172.21.15.52:0/2703230611 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0cc15f070 con 0x7fb0cc070c10 2024-05-04T12:18:52.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d32d6640 1 -- 172.21.15.52:0/2703230611 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0cc15f730 con 0x7fb0cc070c10 2024-05-04T12:18:52.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0c1ffb640 1 -- 172.21.15.52:0/2703230611 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb0c40bf020 con 0x7fb0cc070c10 2024-05-04T12:18:52.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0c1ffb640 1 --2- 172.21.15.52:0/2703230611 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb0b00678c0 0x7fb0b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:52.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0c1ffb640 1 -- 172.21.15.52:0/2703230611 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb0c4120cd0 con 0x7fb0cc070c10 2024-05-04T12:18:52.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d0871640 1 --2- 172.21.15.52:0/2703230611 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb0b00678c0 0x7fb0b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:52.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.949+0000 7fb0d0871640 1 --2- 172.21.15.52:0/2703230611 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb0b00678c0 0x7fb0b0069d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc15fac0 tx=0x7fb0bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:52.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:52.953+0000 7fb0c1ffb640 1 -- 172.21.15.52:0/2703230611 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb0cc15f730 con 0x7fb0cc070c10 2024-05-04T12:18:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.117+0000 7fb0d32d6640 1 -- 172.21.15.52:0/2703230611 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb0cc062e50 con 0x7fb0cc070c10 2024-05-04T12:18:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.117+0000 7fb0c1ffb640 1 -- 172.21.15.52:0/2703230611 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fb0c40c6030 con 0x7fb0cc070c10 2024-05-04T12:18:53.121 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:18:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.121+0000 7fb0d32d6640 1 -- 172.21.15.52:0/2703230611 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb0b00678c0 msgr2=0x7fb0b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.121+0000 7fb0d32d6640 1 --2- 172.21.15.52:0/2703230611 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb0b00678c0 0x7fb0b0069d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc15fac0 tx=0x7fb0bc008040 comp rx=0 tx=0).stop 2024-05-04T12:18:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.121+0000 7fb0d32d6640 1 -- 172.21.15.52:0/2703230611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0cc070c10 msgr2=0x7fb0cc1478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.121+0000 7fb0d32d6640 1 --2- 172.21.15.52:0/2703230611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0cc070c10 0x7fb0cc1478d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40af040 tx=0x7fb0c4019790 comp rx=0 tx=0).stop 2024-05-04T12:18:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.121+0000 7fb0d32d6640 1 -- 172.21.15.52:0/2703230611 shutdown_connections 2024-05-04T12:18:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.121+0000 7fb0d32d6640 1 --2- 172.21.15.52:0/2703230611 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb0b00678c0 0x7fb0b0069d80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.121+0000 7fb0d32d6640 1 --2- 172.21.15.52:0/2703230611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0cc162660 0x7fb0cc071d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.121+0000 7fb0d32d6640 1 --2- 172.21.15.52:0/2703230611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0cc15e180 0x7fb0cc160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.121+0000 7fb0d32d6640 1 --2- 172.21.15.52:0/2703230611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0cc070c10 0x7fb0cc1478d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.121+0000 7fb0d32d6640 1 -- 172.21.15.52:0/2703230611 >> 172.21.15.52:0/2703230611 conn(0x7fb0cc0d12c0 msgr2=0x7fb0cc15bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:53.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.121+0000 7fb0d32d6640 1 -- 172.21.15.52:0/2703230611 shutdown_connections 2024-05-04T12:18:53.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.121+0000 7fb0d32d6640 1 -- 172.21.15.52:0/2703230611 wait complete. 2024-05-04T12:18:53.139 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:18:53.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.265+0000 7f8657d13640 1 Processor -- start 2024-05-04T12:18:53.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.265+0000 7f8657d13640 1 -- start start 2024-05-04T12:18:53.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f8657d13640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8650143b80 0x7f8650144000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f8657d13640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f865006c6c0 0x7f865006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f8657d13640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f865015ed70 con 0x7f8650141560 2024-05-04T12:18:53.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f8657d13640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f8650071ed0 con 0x7f8650143b80 2024-05-04T12:18:53.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f8657d13640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f8650144730 con 0x7f865006c6c0 2024-05-04T12:18:53.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f86552ae640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8650143b80 0x7f8650144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f8655aaf640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f8650141560 0x7f8650072360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55172/0 (socket says 172.21.15.52:55172) 2024-05-04T12:18:53.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f8655aaf640 1 -- 172.21.15.52:0/3285770965 learned_addr learned my addr 172.21.15.52:0/3285770965 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:53.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f86562b0640 1 --2- 172.21.15.52:0/3285770965 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f865006c6c0 0x7f865006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f86552ae640 1 -- 172.21.15.52:0/3285770965 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f865006c6c0 msgr2=0x7f865006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:53.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f86552ae640 1 --2- 172.21.15.52:0/3285770965 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f865006c6c0 0x7f865006cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f86552ae640 1 -- 172.21.15.52:0/3285770965 >> v1:172.21.15.52:6789/0 conn(0x7f8650141560 legacy=0x7f8650072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f8654aad640 1 -- 172.21.15.52:0/3285770965 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3745414831 0 0) 0x7f865015ed70 con 0x7f8650141560 2024-05-04T12:18:53.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f86552ae640 1 -- 172.21.15.52:0/3285770965 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8650071880 con 0x7f8650143b80 2024-05-04T12:18:53.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f86552ae640 1 --2- 172.21.15.52:0/3285770965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8650143b80 0x7f8650144000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f86400097c0 tx=0x7f864001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7c9c32ddf5aee1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:53.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f8654aad640 1 -- 172.21.15.52:0/3285770965 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86400b0070 con 0x7f8650143b80 2024-05-04T12:18:53.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f8654aad640 1 -- 172.21.15.52:0/3285770965 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86400ab600 con 0x7f8650143b80 2024-05-04T12:18:53.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.269+0000 7f8654aad640 1 -- 172.21.15.52:0/3285770965 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86400ab9d0 con 0x7f8650143b80 2024-05-04T12:18:53.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 -- 172.21.15.52:0/3285770965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8650143b80 msgr2=0x7f8650144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 --2- 172.21.15.52:0/3285770965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8650143b80 0x7f8650144000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f86400097c0 tx=0x7f864001c4a0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 -- 172.21.15.52:0/3285770965 shutdown_connections 2024-05-04T12:18:53.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 --2- 172.21.15.52:0/3285770965 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f865006c6c0 0x7f865006cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 --2- 172.21.15.52:0/3285770965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8650143b80 0x7f8650144000 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 -- 172.21.15.52:0/3285770965 >> 172.21.15.52:0/3285770965 conn(0x7f865006b210 msgr2=0x7f865006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:53.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 -- 172.21.15.52:0/3285770965 shutdown_connections 2024-05-04T12:18:53.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 -- 172.21.15.52:0/3285770965 wait complete. 2024-05-04T12:18:53.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 Processor -- start 2024-05-04T12:18:53.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 -- start start 2024-05-04T12:18:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f865006c6c0 0x7f865013db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8650141560 0x7f8650154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8650143b80 0x7f86500aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f865016b570 con 0x7f8650141560 2024-05-04T12:18:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f865016b3f0 con 0x7f8650143b80 2024-05-04T12:18:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8657d13640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f865016b6f0 con 0x7f865006c6c0 2024-05-04T12:18:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f86552ae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8650141560 0x7f8650154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f86552ae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8650141560 0x7f8650154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53822/0 (socket says 172.21.15.52:53822) 2024-05-04T12:18:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f86552ae640 1 -- 172.21.15.52:0/668849852 learned_addr learned my addr 172.21.15.52:0/668849852 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f8655aaf640 1 --2- 172.21.15.52:0/668849852 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f865006c6c0 0x7f865013db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f86562b0640 1 --2- 172.21.15.52:0/668849852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8650143b80 0x7f86500aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f86552ae640 1 -- 172.21.15.52:0/668849852 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f865006c6c0 msgr2=0x7f865013db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f86552ae640 1 --2- 172.21.15.52:0/668849852 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f865006c6c0 0x7f865013db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f86552ae640 1 -- 172.21.15.52:0/668849852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8650143b80 msgr2=0x7f86500aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f86552ae640 1 --2- 172.21.15.52:0/668849852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8650143b80 0x7f86500aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f86552ae640 1 -- 172.21.15.52:0/668849852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865013e0d0 con 0x7f8650141560 2024-05-04T12:18:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f86552ae640 1 --2- 172.21.15.52:0/668849852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8650141560 0x7f8650154560 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f864001ab50 tx=0x7f8640019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f86467fc640 1 -- 172.21.15.52:0/668849852 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86400b0040 con 0x7f8650141560 2024-05-04T12:18:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.273+0000 7f86467fc640 1 -- 172.21.15.52:0/668849852 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86400be070 con 0x7f8650141560 2024-05-04T12:18:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.277+0000 7f8657d13640 1 -- 172.21.15.52:0/668849852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86500af490 con 0x7f8650141560 2024-05-04T12:18:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.277+0000 7f86467fc640 1 -- 172.21.15.52:0/668849852 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86400b78a0 con 0x7f8650141560 2024-05-04T12:18:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.277+0000 7f8657d13640 1 -- 172.21.15.52:0/668849852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865013d0b0 con 0x7f8650141560 2024-05-04T12:18:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.277+0000 7f8657d13640 1 -- 172.21.15.52:0/668849852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8650062e50 con 0x7f8650141560 2024-05-04T12:18:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.277+0000 7f86467fc640 1 -- 172.21.15.52:0/668849852 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f86400b3040 con 0x7f8650141560 2024-05-04T12:18:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.277+0000 7f86467fc640 1 --2- 172.21.15.52:0/668849852 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f86340678c0 0x7f8634069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.277+0000 7f8655aaf640 1 --2- 172.21.15.52:0/668849852 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f86340678c0 0x7f8634069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.277+0000 7f86467fc640 1 -- 172.21.15.52:0/668849852 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8640120bb0 con 0x7f8650141560 2024-05-04T12:18:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.277+0000 7f8655aaf640 1 --2- 172.21.15.52:0/668849852 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f86340678c0 0x7f8634069d80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f864c000f70 tx=0x7f864c007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:53.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.281+0000 7f86467fc640 1 -- 172.21.15.52:0/668849852 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8640016090 con 0x7f8650141560 2024-05-04T12:18:53.405 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:18:53.408 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:18:53.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.441+0000 7f8657d13640 1 -- 172.21.15.52:0/668849852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8650075320 con 0x7f8650141560 2024-05-04T12:18:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.445+0000 7f86467fc640 1 -- 172.21.15.52:0/668849852 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f86400f17f0 con 0x7f8650141560 2024-05-04T12:18:53.447 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:18:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.445+0000 7f8657d13640 1 -- 172.21.15.52:0/668849852 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f86340678c0 msgr2=0x7f8634069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.445+0000 7f8657d13640 1 --2- 172.21.15.52:0/668849852 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f86340678c0 0x7f8634069d80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f864c000f70 tx=0x7f864c007c10 comp rx=0 tx=0).stop 2024-05-04T12:18:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.445+0000 7f8657d13640 1 -- 172.21.15.52:0/668849852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8650141560 msgr2=0x7f8650154560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.445+0000 7f8657d13640 1 --2- 172.21.15.52:0/668849852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8650141560 0x7f8650154560 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f864001ab50 tx=0x7f8640019240 comp rx=0 tx=0).stop 2024-05-04T12:18:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.445+0000 7f8657d13640 1 -- 172.21.15.52:0/668849852 shutdown_connections 2024-05-04T12:18:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.445+0000 7f8657d13640 1 --2- 172.21.15.52:0/668849852 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f86340678c0 0x7f8634069d80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.445+0000 7f8657d13640 1 --2- 172.21.15.52:0/668849852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8650143b80 0x7f86500aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.445+0000 7f8657d13640 1 --2- 172.21.15.52:0/668849852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8650141560 0x7f8650154560 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.445+0000 7f8657d13640 1 --2- 172.21.15.52:0/668849852 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f865006c6c0 0x7f865013db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.445+0000 7f8657d13640 1 -- 172.21.15.52:0/668849852 >> 172.21.15.52:0/668849852 conn(0x7f865006b210 msgr2=0x7f86500b2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.445+0000 7f8657d13640 1 -- 172.21.15.52:0/668849852 shutdown_connections 2024-05-04T12:18:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.445+0000 7f8657d13640 1 -- 172.21.15.52:0/668849852 wait complete. 2024-05-04T12:18:53.466 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:18:53.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e8db8640 1 Processor -- start 2024-05-04T12:18:53.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e8db8640 1 -- start start 2024-05-04T12:18:53.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e8db8640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc4e4141010 0x7fc4e40d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e8db8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc4e40d2540 0x7fc4e41418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e8db8640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e40d29c0 con 0x7fc4e4144570 2024-05-04T12:18:53.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e8db8640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc4e40d2ca0 con 0x7fc4e40d2540 2024-05-04T12:18:53.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e8db8640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e40d2f80 con 0x7fc4e4141010 2024-05-04T12:18:53.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e2d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc4e40d2540 0x7fc4e41418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e2d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc4e40d2540 0x7fc4e41418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53838/0 (socket says 172.21.15.52:53838) 2024-05-04T12:18:53.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e2d9d640 1 -- 172.21.15.52:0/3879318684 learned_addr learned my addr 172.21.15.52:0/3879318684 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:53.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e2d9d640 1 -- 172.21.15.52:0/3879318684 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc4e4141010 msgr2=0x7fc4e40d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:53.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e259c640 1 --2- 172.21.15.52:0/3879318684 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc4e4141010 0x7fc4e40d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e2d9d640 1 --2- 172.21.15.52:0/3879318684 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc4e4141010 0x7fc4e40d2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e2d9d640 1 -- 172.21.15.52:0/3879318684 >> v1:172.21.15.111:6789/0 conn(0x7fc4e4144570 legacy=0x7fc4e4143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:53.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e2d9d640 1 -- 172.21.15.52:0/3879318684 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e40d3260 con 0x7fc4e40d2540 2024-05-04T12:18:53.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.637+0000 7fc4e259c640 1 --2- 172.21.15.52:0/3879318684 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc4e4141010 0x7fc4e40d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:53.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e2d9d640 1 --2- 172.21.15.52:0/3879318684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4e40d2540 0x7fc4e41418b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009870 tx=0x7fc4d401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b944d8fdccc404 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:53.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e159a640 1 -- 172.21.15.52:0/3879318684 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d40ab640 con 0x7fc4e40d2540 2024-05-04T12:18:53.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e159a640 1 -- 172.21.15.52:0/3879318684 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d40b0070 con 0x7fc4e40d2540 2024-05-04T12:18:53.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e159a640 1 -- 172.21.15.52:0/3879318684 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d40aba00 con 0x7fc4e40d2540 2024-05-04T12:18:53.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 -- 172.21.15.52:0/3879318684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4e40d2540 msgr2=0x7fc4e41418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 --2- 172.21.15.52:0/3879318684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4e40d2540 0x7fc4e41418b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009870 tx=0x7fc4d401c460 comp rx=0 tx=0).stop 2024-05-04T12:18:53.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 -- 172.21.15.52:0/3879318684 shutdown_connections 2024-05-04T12:18:53.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 --2- 172.21.15.52:0/3879318684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4e40d2540 0x7fc4e41418b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 --2- 172.21.15.52:0/3879318684 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc4e4141010 0x7fc4e40d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 -- 172.21.15.52:0/3879318684 >> 172.21.15.52:0/3879318684 conn(0x7fc4e4072be0 msgr2=0x7fc4e40d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:53.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 -- 172.21.15.52:0/3879318684 shutdown_connections 2024-05-04T12:18:53.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 -- 172.21.15.52:0/3879318684 wait complete. 2024-05-04T12:18:53.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 Processor -- start 2024-05-04T12:18:53.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 -- start start 2024-05-04T12:18:53.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4e40d2540 0x7fc4e413f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc4e4141010 0x7fc4e4152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4e4144570 0x7fc4e4155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4142600 con 0x7fc4e40d2540 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4144260 con 0x7fc4e4144570 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e8db8640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e4142780 con 0x7fc4e4141010 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4e40d2540 0x7fc4e413f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4e40d2540 0x7fc4e413f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53840/0 (socket says 172.21.15.52:53840) 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.641+0000 7fc4e259c640 1 -- 172.21.15.52:0/1738116895 learned_addr learned my addr 172.21.15.52:0/1738116895 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e1d9b640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc4e4141010 0x7fc4e4152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e2d9d640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4e4144570 0x7fc4e4155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e259c640 1 -- 172.21.15.52:0/1738116895 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc4e4141010 msgr2=0x7fc4e4152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e259c640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc4e4141010 0x7fc4e4152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e259c640 1 -- 172.21.15.52:0/1738116895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4e4144570 msgr2=0x7fc4e4155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e259c640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4e4144570 0x7fc4e4155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e259c640 1 -- 172.21.15.52:0/1738116895 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e413f610 con 0x7fc4e40d2540 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e2d9d640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4e4144570 0x7fc4e4155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:53.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e1d9b640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc4e4141010 0x7fc4e4152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:53.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e259c640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4e40d2540 0x7fc4e413f0d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00c7c0 tx=0x7fc4dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:53.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4daffd640 1 -- 172.21.15.52:0/1738116895 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc007a30 con 0x7fc4e40d2540 2024-05-04T12:18:53.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4daffd640 1 -- 172.21.15.52:0/1738116895 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4dc007bd0 con 0x7fc4e40d2540 2024-05-04T12:18:53.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4daffd640 1 -- 172.21.15.52:0/1738116895 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0113f0 con 0x7fc4e40d2540 2024-05-04T12:18:53.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e8db8640 1 -- 172.21.15.52:0/1738116895 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e4153900 con 0x7fc4e40d2540 2024-05-04T12:18:53.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e8db8640 1 -- 172.21.15.52:0/1738116895 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e4153ec0 con 0x7fc4e40d2540 2024-05-04T12:18:53.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e8db8640 1 -- 172.21.15.52:0/1738116895 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e4062e50 con 0x7fc4e40d2540 2024-05-04T12:18:53.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4daffd640 1 -- 172.21.15.52:0/1738116895 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc4dc011590 con 0x7fc4e40d2540 2024-05-04T12:18:53.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4daffd640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc4c40678f0 0x7fc4c4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e1d9b640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc4c40678f0 0x7fc4c4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4daffd640 1 -- 172.21.15.52:0/1738116895 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc4dc085e30 con 0x7fc4e40d2540 2024-05-04T12:18:53.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.645+0000 7fc4e1d9b640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc4c40678f0 0x7fc4c4069db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40aa940 tx=0x7fc4cc0073a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:53.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.649+0000 7fc4daffd640 1 -- 172.21.15.52:0/1738116895 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc4dc015020 con 0x7fc4e40d2540 2024-05-04T12:18:53.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.809+0000 7fc4e8db8640 1 -- 172.21.15.52:0/1738116895 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc4e4075320 con 0x7fc4e40d2540 2024-05-04T12:18:53.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.813+0000 7fc4daffd640 1 -- 172.21.15.52:0/1738116895 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fc4dc056960 con 0x7fc4e40d2540 2024-05-04T12:18:53.815 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:18:53.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.813+0000 7fc4e8db8640 1 -- 172.21.15.52:0/1738116895 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc4c40678f0 msgr2=0x7fc4c4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.813+0000 7fc4e8db8640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc4c40678f0 0x7fc4c4069db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40aa940 tx=0x7fc4cc0073a0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.813+0000 7fc4e8db8640 1 -- 172.21.15.52:0/1738116895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4e40d2540 msgr2=0x7fc4e413f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.813+0000 7fc4e8db8640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4e40d2540 0x7fc4e413f0d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00c7c0 tx=0x7fc4dc00cc80 comp rx=0 tx=0).stop 2024-05-04T12:18:53.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.813+0000 7fc4e8db8640 1 -- 172.21.15.52:0/1738116895 shutdown_connections 2024-05-04T12:18:53.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.813+0000 7fc4e8db8640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc4c40678f0 0x7fc4c4069db0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.813+0000 7fc4e8db8640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4e4144570 0x7fc4e4155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.813+0000 7fc4e8db8640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc4e4141010 0x7fc4e4152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.813+0000 7fc4e8db8640 1 --2- 172.21.15.52:0/1738116895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4e40d2540 0x7fc4e413f0d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.813+0000 7fc4e8db8640 1 -- 172.21.15.52:0/1738116895 >> 172.21.15.52:0/1738116895 conn(0x7fc4e4072be0 msgr2=0x7fc4e415d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:53.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.813+0000 7fc4e8db8640 1 -- 172.21.15.52:0/1738116895 shutdown_connections 2024-05-04T12:18:53.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.813+0000 7fc4e8db8640 1 -- 172.21.15.52:0/1738116895 wait complete. 2024-05-04T12:18:53.833 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2024-05-04T12:18:53.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.937+0000 7f4c28164640 1 Processor -- start 2024-05-04T12:18:53.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.937+0000 7f4c28164640 1 -- start start 2024-05-04T12:18:53.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c28164640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4c2015bf50 0x7f4c2015c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c28164640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c200cd260 0x7f4c2006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c28164640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c2006d1b0 con 0x7f4c200aa0c0 2024-05-04T12:18:53.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c28164640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f4c2006d490 con 0x7f4c2015bf50 2024-05-04T12:18:53.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c28164640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2006d770 con 0x7f4c200cd260 2024-05-04T12:18:53.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c25f00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4c2015bf50 0x7f4c2015c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c256ff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f4c200aa0c0 0x7f4c200ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55178/0 (socket says 172.21.15.52:55178) 2024-05-04T12:18:53.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c256ff640 1 -- 172.21.15.52:0/978264696 learned_addr learned my addr 172.21.15.52:0/978264696 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:53.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c26701640 1 --2- 172.21.15.52:0/978264696 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c200cd260 0x7f4c2006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c24efe640 1 -- 172.21.15.52:0/978264696 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2216264015 0 0) 0x7f4c2006d1b0 con 0x7f4c200aa0c0 2024-05-04T12:18:53.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c24efe640 1 -- 172.21.15.52:0/978264696 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c0c003530 con 0x7f4c200aa0c0 2024-05-04T12:18:53.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c25f00640 1 -- 172.21.15.52:0/978264696 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c200cd260 msgr2=0x7f4c2006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:53.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c25f00640 1 --2- 172.21.15.52:0/978264696 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c200cd260 0x7f4c2006e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c25f00640 1 -- 172.21.15.52:0/978264696 >> v1:172.21.15.52:6789/0 conn(0x7f4c200aa0c0 legacy=0x7f4c200ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c25f00640 1 -- 172.21.15.52:0/978264696 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2006da50 con 0x7f4c2015bf50 2024-05-04T12:18:53.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c25f00640 1 --2- 172.21.15.52:0/978264696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c2015bf50 0x7f4c2015c330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0097c0 tx=0x7f4c1c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d5685d8219b74e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:53.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c24efe640 1 -- 172.21.15.52:0/978264696 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0b2070 con 0x7f4c2015bf50 2024-05-04T12:18:53.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c24efe640 1 -- 172.21.15.52:0/978264696 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c1c0ad680 con 0x7f4c2015bf50 2024-05-04T12:18:53.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.941+0000 7f4c24efe640 1 -- 172.21.15.52:0/978264696 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0ada50 con 0x7f4c2015bf50 2024-05-04T12:18:53.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 -- 172.21.15.52:0/978264696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c2015bf50 msgr2=0x7f4c2015c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 --2- 172.21.15.52:0/978264696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c2015bf50 0x7f4c2015c330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0097c0 tx=0x7f4c1c01c470 comp rx=0 tx=0).stop 2024-05-04T12:18:53.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 -- 172.21.15.52:0/978264696 shutdown_connections 2024-05-04T12:18:53.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 --2- 172.21.15.52:0/978264696 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c200cd260 0x7f4c2006e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 --2- 172.21.15.52:0/978264696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c2015bf50 0x7f4c2015c330 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 -- 172.21.15.52:0/978264696 >> 172.21.15.52:0/978264696 conn(0x7f4c200b14a0 msgr2=0x7f4c200b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:53.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 -- 172.21.15.52:0/978264696 shutdown_connections 2024-05-04T12:18:53.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 -- 172.21.15.52:0/978264696 wait complete. 2024-05-04T12:18:53.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 Processor -- start 2024-05-04T12:18:53.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 -- start start 2024-05-04T12:18:53.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c200aa0c0 0x7f4c201570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c200cd260 0x7f4c20070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c2015bf50 0x7f4c2014a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4c201691b0 con 0x7f4c2015bf50 2024-05-04T12:18:53.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4c20169030 con 0x7f4c200aa0c0 2024-05-04T12:18:53.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4c20169330 con 0x7f4c200cd260 2024-05-04T12:18:53.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c26701640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c2015bf50 0x7f4c2014a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c26701640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c2015bf50 0x7f4c2014a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53850/0 (socket says 172.21.15.52:53850) 2024-05-04T12:18:53.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c26701640 1 -- 172.21.15.52:0/2175418790 learned_addr learned my addr 172.21.15.52:0/2175418790 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c256ff640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c200cd260 0x7f4c20070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c25f00640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c200aa0c0 0x7f4c201570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c26701640 1 -- 172.21.15.52:0/2175418790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c200cd260 msgr2=0x7f4c20070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c26701640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c200cd260 0x7f4c20070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c26701640 1 -- 172.21.15.52:0/2175418790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c200aa0c0 msgr2=0x7f4c201570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c26701640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c200aa0c0 0x7f4c201570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c26701640 1 -- 172.21.15.52:0/2175418790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c200cfe90 con 0x7f4c2015bf50 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c256ff640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c200cd260 0x7f4c20070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c25f00640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c200aa0c0 0x7f4c201570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c26701640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c2015bf50 0x7f4c2014a910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4c18007de0 tx=0x7f4c1800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c167fc640 1 -- 172.21.15.52:0/2175418790 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c18014620 con 0x7f4c2015bf50 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c167fc640 1 -- 172.21.15.52:0/2175418790 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c18019070 con 0x7f4c2015bf50 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 -- 172.21.15.52:0/2175418790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c20151da0 con 0x7f4c2015bf50 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c167fc640 1 -- 172.21.15.52:0/2175418790 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c18014a20 con 0x7f4c2015bf50 2024-05-04T12:18:53.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.945+0000 7f4c28164640 1 -- 172.21.15.52:0/2175418790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c20157600 con 0x7f4c2015bf50 2024-05-04T12:18:53.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.949+0000 7f4c28164640 1 -- 172.21.15.52:0/2175418790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c20062e50 con 0x7f4c2015bf50 2024-05-04T12:18:53.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.949+0000 7f4c167fc640 1 -- 172.21.15.52:0/2175418790 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4c18014bc0 con 0x7f4c2015bf50 2024-05-04T12:18:53.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.949+0000 7f4c167fc640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4c040678f0 0x7f4c04069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:53.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.949+0000 7f4c167fc640 1 -- 172.21.15.52:0/2175418790 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4c18086520 con 0x7f4c2015bf50 2024-05-04T12:18:53.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.949+0000 7f4c25f00640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4c040678f0 0x7f4c04069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:53.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.949+0000 7f4c25f00640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4c040678f0 0x7f4c04069db0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0af000 tx=0x7f4c1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:53.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:53.953+0000 7f4c167fc640 1 -- 172.21.15.52:0/2175418790 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4c180570f0 con 0x7f4c2015bf50 2024-05-04T12:18:54.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:54.165+0000 7f4c28164640 1 -- 172.21.15.52:0/2175418790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) -- 0x7f4c20075320 con 0x7f4c2015bf50 2024-05-04T12:18:55.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:54.997+0000 7f4c167fc640 1 -- 172.21.15.52:0/2175418790 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v52) ==== 147+0+0 (secure 0 0 0) 0x7f4c180569f0 con 0x7f4c2015bf50 2024-05-04T12:18:55.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.001+0000 7f4c28164640 1 -- 172.21.15.52:0/2175418790 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4c040678f0 msgr2=0x7f4c04069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:55.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.001+0000 7f4c28164640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4c040678f0 0x7f4c04069db0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0af000 tx=0x7f4c1c008040 comp rx=0 tx=0).stop 2024-05-04T12:18:55.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.001+0000 7f4c28164640 1 -- 172.21.15.52:0/2175418790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c2015bf50 msgr2=0x7f4c2014a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:55.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.001+0000 7f4c28164640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c2015bf50 0x7f4c2014a910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4c18007de0 tx=0x7f4c1800e4b0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.001+0000 7f4c28164640 1 -- 172.21.15.52:0/2175418790 shutdown_connections 2024-05-04T12:18:55.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.001+0000 7f4c28164640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4c040678f0 0x7f4c04069db0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.001+0000 7f4c28164640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c2015bf50 0x7f4c2014a910 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.001+0000 7f4c28164640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c200cd260 0x7f4c20070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.001+0000 7f4c28164640 1 --2- 172.21.15.52:0/2175418790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c200aa0c0 0x7f4c201570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.005+0000 7f4c28164640 1 -- 172.21.15.52:0/2175418790 >> 172.21.15.52:0/2175418790 conn(0x7f4c200b14a0 msgr2=0x7f4c20159150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:55.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.005+0000 7f4c28164640 1 -- 172.21.15.52:0/2175418790 shutdown_connections 2024-05-04T12:18:55.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.005+0000 7f4c28164640 1 -- 172.21.15.52:0/2175418790 wait complete. 2024-05-04T12:18:55.029 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-04T12:18:55.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff2541d0640 1 Processor -- start 2024-05-04T12:18:55.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff2541d0640 1 -- start start 2024-05-04T12:18:55.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff2541d0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff24c15c590 0x7ff24c0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:55.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff2541d0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff24c0cccd0 0x7ff24c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:55.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff2541d0640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff24c06dbf0 con 0x7ff24c145960 2024-05-04T12:18:55.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff2541d0640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ff24c0cd140 con 0x7ff24c15c590 2024-05-04T12:18:55.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff2541d0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7ff24c06cfe0 con 0x7ff24c0cccd0 2024-05-04T12:18:55.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff25176b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff24c15c590 0x7ff24c0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff251f6c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7ff24c145960 0x7ff24c160590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55186/0 (socket says 172.21.15.52:55186) 2024-05-04T12:18:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff25176b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff24c15c590 0x7ff24c0cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53854/0 (socket says 172.21.15.52:53854) 2024-05-04T12:18:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff251f6c640 1 -- 172.21.15.52:0/3348153646 learned_addr learned my addr 172.21.15.52:0/3348153646 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff25176b640 1 -- 172.21.15.52:0/3348153646 >> v2:172.21.15.111:3300/0 conn(0x7ff24c0cccd0 msgr2=0x7ff24c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff25276d640 1 --2- 172.21.15.52:0/3348153646 >> v2:172.21.15.111:3300/0 conn(0x7ff24c0cccd0 0x7ff24c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff250f6a640 1 -- 172.21.15.52:0/3348153646 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 369818391 0 0) 0x7ff24c06dbf0 con 0x7ff24c145960 2024-05-04T12:18:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff25176b640 1 --2- 172.21.15.52:0/3348153646 >> v2:172.21.15.111:3300/0 conn(0x7ff24c0cccd0 0x7ff24c06e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff25176b640 1 -- 172.21.15.52:0/3348153646 >> v1:172.21.15.52:6789/0 conn(0x7ff24c145960 legacy=0x7ff24c160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff25176b640 1 -- 172.21.15.52:0/3348153646 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c06d2c0 con 0x7ff24c15c590 2024-05-04T12:18:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.161+0000 7ff25276d640 1 --2- 172.21.15.52:0/3348153646 >> v2:172.21.15.111:3300/0 conn(0x7ff24c0cccd0 0x7ff24c06e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:55.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.165+0000 7ff25176b640 1 --2- 172.21.15.52:0/3348153646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff24c15c590 0x7ff24c0cc790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0af290 tx=0x7ff23c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f9d0a28dfce586 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:55.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.165+0000 7ff250f6a640 1 -- 172.21.15.52:0/3348153646 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0b1070 con 0x7ff24c15c590 2024-05-04T12:18:55.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.165+0000 7ff250f6a640 1 -- 172.21.15.52:0/3348153646 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff23c0ac620 con 0x7ff24c15c590 2024-05-04T12:18:55.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.165+0000 7ff250f6a640 1 -- 172.21.15.52:0/3348153646 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0ac9f0 con 0x7ff24c15c590 2024-05-04T12:18:55.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.165+0000 7ff2541d0640 1 -- 172.21.15.52:0/3348153646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff24c15c590 msgr2=0x7ff24c0cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:55.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.165+0000 7ff2541d0640 1 --2- 172.21.15.52:0/3348153646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff24c15c590 0x7ff24c0cc790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0af290 tx=0x7ff23c01c3c0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.165+0000 7ff2541d0640 1 -- 172.21.15.52:0/3348153646 shutdown_connections 2024-05-04T12:18:55.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.165+0000 7ff2541d0640 1 --2- 172.21.15.52:0/3348153646 >> v2:172.21.15.111:3300/0 conn(0x7ff24c0cccd0 0x7ff24c06e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.165+0000 7ff2541d0640 1 --2- 172.21.15.52:0/3348153646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff24c15c590 0x7ff24c0cc790 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.165+0000 7ff2541d0640 1 -- 172.21.15.52:0/3348153646 >> 172.21.15.52:0/3348153646 conn(0x7ff24c067730 msgr2=0x7ff24c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:55.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.165+0000 7ff2541d0640 1 -- 172.21.15.52:0/3348153646 shutdown_connections 2024-05-04T12:18:55.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.165+0000 7ff2541d0640 1 -- 172.21.15.52:0/3348153646 wait complete. 2024-05-04T12:18:55.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.165+0000 7ff2541d0640 1 Processor -- start 2024-05-04T12:18:55.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.165+0000 7ff2541d0640 1 -- start start 2024-05-04T12:18:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2541d0640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff24c0cccd0 0x7ff24c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2541d0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff24c145960 0x7ff24c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2541d0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff24c15c590 0x7ff24c15cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2541d0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c06cdc0 con 0x7ff24c15c590 2024-05-04T12:18:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2541d0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c06cc40 con 0x7ff24c145960 2024-05-04T12:18:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2541d0640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7ff24c145d40 con 0x7ff24c0cccd0 2024-05-04T12:18:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff25276d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff24c15c590 0x7ff24c15cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff25276d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff24c15c590 0x7ff24c15cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53862/0 (socket says 172.21.15.52:53862) 2024-05-04T12:18:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff25276d640 1 -- 172.21.15.52:0/4006840945 learned_addr learned my addr 172.21.15.52:0/4006840945 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff25276d640 1 -- 172.21.15.52:0/4006840945 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff24c0cccd0 msgr2=0x7ff24c15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff251f6c640 1 --2- 172.21.15.52:0/4006840945 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff24c0cccd0 0x7ff24c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff25176b640 1 --2- 172.21.15.52:0/4006840945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff24c145960 0x7ff24c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff25276d640 1 --2- 172.21.15.52:0/4006840945 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff24c0cccd0 0x7ff24c15b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff25276d640 1 -- 172.21.15.52:0/4006840945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff24c145960 msgr2=0x7ff24c15c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff25276d640 1 --2- 172.21.15.52:0/4006840945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff24c145960 0x7ff24c15c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff25276d640 1 -- 172.21.15.52:0/4006840945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c071120 con 0x7ff24c15c590 2024-05-04T12:18:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff25276d640 1 --2- 172.21.15.52:0/4006840945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff24c15c590 0x7ff24c15cee0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff24400cd50 tx=0x7ff24400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2427fc640 1 -- 172.21.15.52:0/4006840945 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24400f680 con 0x7ff24c15c590 2024-05-04T12:18:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2427fc640 1 -- 172.21.15.52:0/4006840945 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff24400f840 con 0x7ff24c15c590 2024-05-04T12:18:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2541d0640 1 -- 172.21.15.52:0/4006840945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24c071400 con 0x7ff24c15c590 2024-05-04T12:18:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2427fc640 1 -- 172.21.15.52:0/4006840945 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24400fc30 con 0x7ff24c15c590 2024-05-04T12:18:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2541d0640 1 -- 172.21.15.52:0/4006840945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff24c0719c0 con 0x7ff24c15c590 2024-05-04T12:18:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2541d0640 1 -- 172.21.15.52:0/4006840945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff24c062e50 con 0x7ff24c15c590 2024-05-04T12:18:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2427fc640 1 -- 172.21.15.52:0/4006840945 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff2440213f0 con 0x7ff24c15c590 2024-05-04T12:18:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2427fc640 1 --2- 172.21.15.52:0/4006840945 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff2300678c0 0x7ff230069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff2427fc640 1 -- 172.21.15.52:0/4006840945 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff2440869e0 con 0x7ff24c15c590 2024-05-04T12:18:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff251f6c640 1 --2- 172.21.15.52:0/4006840945 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff2300678c0 0x7ff230069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:55.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.169+0000 7ff251f6c640 1 --2- 172.21.15.52:0/4006840945 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff2300678c0 0x7ff230069d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff248000f70 tx=0x7ff248009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:55.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.173+0000 7ff2427fc640 1 -- 172.21.15.52:0/4006840945 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff244014030 con 0x7ff24c15c590 2024-05-04T12:18:55.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.337+0000 7ff2541d0640 1 -- 172.21.15.52:0/4006840945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff24c075320 con 0x7ff24c15c590 2024-05-04T12:18:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.341+0000 7ff2427fc640 1 -- 172.21.15.52:0/4006840945 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7ff2440575b0 con 0x7ff24c15c590 2024-05-04T12:18:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.345+0000 7ff2541d0640 1 -- 172.21.15.52:0/4006840945 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff2300678c0 msgr2=0x7ff230069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.345+0000 7ff2541d0640 1 --2- 172.21.15.52:0/4006840945 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff2300678c0 0x7ff230069d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff248000f70 tx=0x7ff248009040 comp rx=0 tx=0).stop 2024-05-04T12:18:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.345+0000 7ff2541d0640 1 -- 172.21.15.52:0/4006840945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff24c15c590 msgr2=0x7ff24c15cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.345+0000 7ff2541d0640 1 --2- 172.21.15.52:0/4006840945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff24c15c590 0x7ff24c15cee0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff24400cd50 tx=0x7ff24400a650 comp rx=0 tx=0).stop 2024-05-04T12:18:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.345+0000 7ff2541d0640 1 -- 172.21.15.52:0/4006840945 shutdown_connections 2024-05-04T12:18:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.345+0000 7ff2541d0640 1 --2- 172.21.15.52:0/4006840945 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff2300678c0 0x7ff230069d80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.345+0000 7ff2541d0640 1 --2- 172.21.15.52:0/4006840945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff24c15c590 0x7ff24c15cee0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.345+0000 7ff2541d0640 1 --2- 172.21.15.52:0/4006840945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff24c145960 0x7ff24c15c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.345+0000 7ff2541d0640 1 --2- 172.21.15.52:0/4006840945 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff24c0cccd0 0x7ff24c15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.345+0000 7ff2541d0640 1 -- 172.21.15.52:0/4006840945 >> 172.21.15.52:0/4006840945 conn(0x7ff24c067730 msgr2=0x7ff24c1603e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:55.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.345+0000 7ff2541d0640 1 -- 172.21.15.52:0/4006840945 shutdown_connections 2024-05-04T12:18:55.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.345+0000 7ff2541d0640 1 -- 172.21.15.52:0/4006840945 wait complete. 2024-05-04T12:18:55.365 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-04T12:18:55.365 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-04T12:18:55.420 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-04T12:18:55.467 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-04T12:18:55.467 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:18:55.467 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T12:18:55.473 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:18:55.473 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T12:18:55.519 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:55.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b9012640 1 Processor -- start 2024-05-04T12:18:55.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b9012640 1 -- start start 2024-05-04T12:18:55.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b9012640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff9b40cc2c0 0x7ff9b415e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:55.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b9012640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff9b415a530 0x7ff9b40ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:55.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b9012640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b415ea30 con 0x7ff9b415a9c0 2024-05-04T12:18:55.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b9012640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b406f080 con 0x7ff9b40cc2c0 2024-05-04T12:18:55.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b9012640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b415ed10 con 0x7ff9b415a530 2024-05-04T12:18:55.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b359e640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7ff9b415a9c0 0x7ff9b4071be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55194/0 (socket says 172.21.15.52:55194) 2024-05-04T12:18:55.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b359e640 1 -- 172.21.15.52:0/4218908633 learned_addr learned my addr 172.21.15.52:0/4218908633 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:55.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b2d9d640 1 --2- 172.21.15.52:0/4218908633 >> v2:172.21.15.52:3300/0 conn(0x7ff9b40cc2c0 0x7ff9b415e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:55.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b259c640 1 --2- 172.21.15.52:0/4218908633 >> v2:172.21.15.111:3300/0 conn(0x7ff9b415a530 0x7ff9b40ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:55.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b2d9d640 1 -- 172.21.15.52:0/4218908633 >> v2:172.21.15.111:3300/0 conn(0x7ff9b415a530 msgr2=0x7ff9b40ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:55.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b2d9d640 1 --2- 172.21.15.52:0/4218908633 >> v2:172.21.15.111:3300/0 conn(0x7ff9b415a530 0x7ff9b40ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b2d9d640 1 -- 172.21.15.52:0/4218908633 >> v1:172.21.15.52:6789/0 conn(0x7ff9b415a9c0 legacy=0x7ff9b4071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:55.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b2d9d640 1 -- 172.21.15.52:0/4218908633 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b415eff0 con 0x7ff9b40cc2c0 2024-05-04T12:18:55.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.697+0000 7ff9b259c640 1 --2- 172.21.15.52:0/4218908633 >> v2:172.21.15.111:3300/0 conn(0x7ff9b415a530 0x7ff9b40ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:55.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b2d9d640 1 --2- 172.21.15.52:0/4218908633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9b40cc2c0 0x7ff9b415e330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff9b415e270 tx=0x7ff9a401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdde9b73cc0f161 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:55.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b1d9b640 1 -- 172.21.15.52:0/4218908633 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a40b0070 con 0x7ff9b40cc2c0 2024-05-04T12:18:55.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b1d9b640 1 -- 172.21.15.52:0/4218908633 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a40ab600 con 0x7ff9b40cc2c0 2024-05-04T12:18:55.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b1d9b640 1 -- 172.21.15.52:0/4218908633 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a40ab9d0 con 0x7ff9b40cc2c0 2024-05-04T12:18:55.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 -- 172.21.15.52:0/4218908633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9b40cc2c0 msgr2=0x7ff9b415e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:55.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 --2- 172.21.15.52:0/4218908633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9b40cc2c0 0x7ff9b415e330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff9b415e270 tx=0x7ff9a401c3f0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 -- 172.21.15.52:0/4218908633 shutdown_connections 2024-05-04T12:18:55.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 --2- 172.21.15.52:0/4218908633 >> v2:172.21.15.111:3300/0 conn(0x7ff9b415a530 0x7ff9b40ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 --2- 172.21.15.52:0/4218908633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9b40cc2c0 0x7ff9b415e330 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 -- 172.21.15.52:0/4218908633 >> 172.21.15.52:0/4218908633 conn(0x7ff9b40b14a0 msgr2=0x7ff9b40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:55.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 -- 172.21.15.52:0/4218908633 shutdown_connections 2024-05-04T12:18:55.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 -- 172.21.15.52:0/4218908633 wait complete. 2024-05-04T12:18:55.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 Processor -- start 2024-05-04T12:18:55.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 -- start start 2024-05-04T12:18:55.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9b40cc2c0 0x7ff9b4154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:55.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9b415a530 0x7ff9b4160d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:55.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b40b2590 0x7ff9b40b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:55.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b406e740 con 0x7ff9b415a530 2024-05-04T12:18:55.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b4072500 con 0x7ff9b40b2590 2024-05-04T12:18:55.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.701+0000 7ff9b9012640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b40cd320 con 0x7ff9b40cc2c0 2024-05-04T12:18:55.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9b415a530 0x7ff9b4160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:55.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9b415a530 0x7ff9b4160d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53880/0 (socket says 172.21.15.52:53880) 2024-05-04T12:18:55.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b259c640 1 -- 172.21.15.52:0/1942462561 learned_addr learned my addr 172.21.15.52:0/1942462561 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:55.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b359e640 1 --2- 172.21.15.52:0/1942462561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b40b2590 0x7ff9b40b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:55.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b2d9d640 1 --2- 172.21.15.52:0/1942462561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9b40cc2c0 0x7ff9b4154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:55.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b259c640 1 -- 172.21.15.52:0/1942462561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9b40cc2c0 msgr2=0x7ff9b4154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:55.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b259c640 1 --2- 172.21.15.52:0/1942462561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9b40cc2c0 0x7ff9b4154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b259c640 1 -- 172.21.15.52:0/1942462561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b40b2590 msgr2=0x7ff9b40b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:55.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b259c640 1 --2- 172.21.15.52:0/1942462561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b40b2590 0x7ff9b40b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b259c640 1 -- 172.21.15.52:0/1942462561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b4155110 con 0x7ff9b415a530 2024-05-04T12:18:55.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b259c640 1 --2- 172.21.15.52:0/1942462561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9b415a530 0x7ff9b4160d90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009fc0 tx=0x7ff9a800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:55.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff99b7fe640 1 -- 172.21.15.52:0/1942462561 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a801a070 con 0x7ff9b415a530 2024-05-04T12:18:55.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff99b7fe640 1 -- 172.21.15.52:0/1942462561 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a8003bc0 con 0x7ff9b415a530 2024-05-04T12:18:55.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b9012640 1 -- 172.21.15.52:0/1942462561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b41612d0 con 0x7ff9b415a530 2024-05-04T12:18:55.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff99b7fe640 1 -- 172.21.15.52:0/1942462561 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8004030 con 0x7ff9b415a530 2024-05-04T12:18:55.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b9012640 1 -- 172.21.15.52:0/1942462561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b41640d0 con 0x7ff9b415a530 2024-05-04T12:18:55.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b9012640 1 -- 172.21.15.52:0/1942462561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b4062e50 con 0x7ff9b415a530 2024-05-04T12:18:55.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff99b7fe640 1 -- 172.21.15.52:0/1942462561 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff9a801d070 con 0x7ff9b415a530 2024-05-04T12:18:55.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff99b7fe640 1 --2- 172.21.15.52:0/1942462561 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff9a00678c0 0x7ff9a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:55.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff99b7fe640 1 -- 172.21.15.52:0/1942462561 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff9a8085ea0 con 0x7ff9b415a530 2024-05-04T12:18:55.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b2d9d640 1 --2- 172.21.15.52:0/1942462561 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff9a00678c0 0x7ff9a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:55.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.705+0000 7ff9b2d9d640 1 --2- 172.21.15.52:0/1942462561 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff9a00678c0 0x7ff9a0069d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff9a40ad000 tx=0x7ff9a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:55.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.709+0000 7ff99b7fe640 1 -- 172.21.15.52:0/1942462561 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff9a8010070 con 0x7ff9b415a530 2024-05-04T12:18:55.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.873+0000 7ff9b9012640 1 -- 172.21.15.52:0/1942462561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9b4075320 con 0x7ff9b415a530 2024-05-04T12:18:55.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.877+0000 7ff99b7fe640 1 -- 172.21.15.52:0/1942462561 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7ff9a8056420 con 0x7ff9b415a530 2024-05-04T12:18:55.879 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:55.879 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.877+0000 7ff9b9012640 1 -- 172.21.15.52:0/1942462561 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff9a00678c0 msgr2=0x7ff9a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.877+0000 7ff9b9012640 1 --2- 172.21.15.52:0/1942462561 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff9a00678c0 0x7ff9a0069d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff9a40ad000 tx=0x7ff9a4008040 comp rx=0 tx=0).stop 2024-05-04T12:18:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.877+0000 7ff9b9012640 1 -- 172.21.15.52:0/1942462561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9b415a530 msgr2=0x7ff9b4160d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.877+0000 7ff9b9012640 1 --2- 172.21.15.52:0/1942462561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9b415a530 0x7ff9b4160d90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009fc0 tx=0x7ff9a800d560 comp rx=0 tx=0).stop 2024-05-04T12:18:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.877+0000 7ff9b9012640 1 -- 172.21.15.52:0/1942462561 shutdown_connections 2024-05-04T12:18:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.877+0000 7ff9b9012640 1 --2- 172.21.15.52:0/1942462561 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff9a00678c0 0x7ff9a0069d80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.877+0000 7ff9b9012640 1 --2- 172.21.15.52:0/1942462561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b40b2590 0x7ff9b40b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.877+0000 7ff9b9012640 1 --2- 172.21.15.52:0/1942462561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9b415a530 0x7ff9b4160d90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.877+0000 7ff9b9012640 1 --2- 172.21.15.52:0/1942462561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9b40cc2c0 0x7ff9b4154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.877+0000 7ff9b9012640 1 -- 172.21.15.52:0/1942462561 >> 172.21.15.52:0/1942462561 conn(0x7ff9b40b14a0 msgr2=0x7ff9b4158100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.877+0000 7ff9b9012640 1 -- 172.21.15.52:0/1942462561 shutdown_connections 2024-05-04T12:18:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:55.877+0000 7ff9b9012640 1 -- 172.21.15.52:0/1942462561 wait complete. 2024-05-04T12:18:55.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:18:55.900 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:18:56.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a38f42640 1 Processor -- start 2024-05-04T12:18:56.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a38f42640 1 -- start start 2024-05-04T12:18:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a38f42640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34144570 0x7f1a34143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a38f42640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1a34143e80 0x7f1a341418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a38f42640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a340d2710 con 0x7f1a34141010 2024-05-04T12:18:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a38f42640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f1a340d29c0 con 0x7f1a34143e80 2024-05-04T12:18:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a38f42640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f1a340d2ca0 con 0x7f1a34144570 2024-05-04T12:18:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a3259c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f1a34141010 0x7f1a340d2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55204/0 (socket says 172.21.15.52:55204) 2024-05-04T12:18:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a3259c640 1 -- 172.21.15.52:0/3577642462 learned_addr learned my addr 172.21.15.52:0/3577642462 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a32d9d640 1 --2- 172.21.15.52:0/3577642462 >> v2:172.21.15.52:3300/0 conn(0x7f1a34143e80 0x7f1a341418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a31d9b640 1 --2- 172.21.15.52:0/3577642462 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34144570 0x7f1a34143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a3159a640 1 -- 172.21.15.52:0/3577642462 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 366769743 0 0) 0x7f1a340d2710 con 0x7f1a34141010 2024-05-04T12:18:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a3159a640 1 -- 172.21.15.52:0/3577642462 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a1c003530 con 0x7f1a34141010 2024-05-04T12:18:56.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a32d9d640 1 -- 172.21.15.52:0/3577642462 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34144570 msgr2=0x7f1a34143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:56.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a32d9d640 1 --2- 172.21.15.52:0/3577642462 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34144570 0x7f1a34143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a32d9d640 1 -- 172.21.15.52:0/3577642462 >> v1:172.21.15.52:6789/0 conn(0x7f1a34141010 legacy=0x7f1a340d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.021+0000 7f1a32d9d640 1 -- 172.21.15.52:0/3577642462 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a340d2f80 con 0x7f1a34143e80 2024-05-04T12:18:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a32d9d640 1 --2- 172.21.15.52:0/3577642462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a34143e80 0x7f1a341418b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009d90 tx=0x7f1a2c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c6761b8fcf9401 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a3159a640 1 -- 172.21.15.52:0/3577642462 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0b2070 con 0x7f1a34143e80 2024-05-04T12:18:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a3159a640 1 -- 172.21.15.52:0/3577642462 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2c0a5890 con 0x7f1a34143e80 2024-05-04T12:18:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a3159a640 1 -- 172.21.15.52:0/3577642462 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0a5c60 con 0x7f1a34143e80 2024-05-04T12:18:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 -- 172.21.15.52:0/3577642462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a34143e80 msgr2=0x7f1a341418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 --2- 172.21.15.52:0/3577642462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a34143e80 0x7f1a341418b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009d90 tx=0x7f1a2c01c8e0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 -- 172.21.15.52:0/3577642462 shutdown_connections 2024-05-04T12:18:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 --2- 172.21.15.52:0/3577642462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a34143e80 0x7f1a341418b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 --2- 172.21.15.52:0/3577642462 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34144570 0x7f1a34143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 -- 172.21.15.52:0/3577642462 >> 172.21.15.52:0/3577642462 conn(0x7f1a34072be0 msgr2=0x7f1a340d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 -- 172.21.15.52:0/3577642462 shutdown_connections 2024-05-04T12:18:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 -- 172.21.15.52:0/3577642462 wait complete. 2024-05-04T12:18:56.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 Processor -- start 2024-05-04T12:18:56.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 -- start start 2024-05-04T12:18:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a34141010 0x7f1a3413a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a34143e80 0x7f1a3415f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34144570 0x7f1a340acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1a34142780 con 0x7f1a34141010 2024-05-04T12:18:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1a34142600 con 0x7f1a34143e80 2024-05-04T12:18:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a38f42640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f1a34142900 con 0x7f1a34144570 2024-05-04T12:18:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a3259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a34141010 0x7f1a3413a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a3259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a34141010 0x7f1a3413a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53906/0 (socket says 172.21.15.52:53906) 2024-05-04T12:18:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a3259c640 1 -- 172.21.15.52:0/4159038114 learned_addr learned my addr 172.21.15.52:0/4159038114 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a32d9d640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34144570 0x7f1a340acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a31d9b640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a34143e80 0x7f1a3415f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a3259c640 1 -- 172.21.15.52:0/4159038114 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34144570 msgr2=0x7f1a340acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a3259c640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34144570 0x7f1a340acaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a3259c640 1 -- 172.21.15.52:0/4159038114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a34143e80 msgr2=0x7f1a3415f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a3259c640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a34143e80 0x7f1a3415f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a3259c640 1 -- 172.21.15.52:0/4159038114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a341398c0 con 0x7f1a34141010 2024-05-04T12:18:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a32d9d640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34144570 0x7f1a340acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a31d9b640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a34143e80 0x7f1a3415f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.025+0000 7f1a3259c640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a34141010 0x7f1a3413a6f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400c3c0 tx=0x7f1a2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.029+0000 7f1a22ffd640 1 -- 172.21.15.52:0/4159038114 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24017070 con 0x7f1a34141010 2024-05-04T12:18:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.029+0000 7f1a22ffd640 1 -- 172.21.15.52:0/4159038114 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a24005cf0 con 0x7f1a34141010 2024-05-04T12:18:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.029+0000 7f1a38f42640 1 -- 172.21.15.52:0/4159038114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a340abfc0 con 0x7f1a34141010 2024-05-04T12:18:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.029+0000 7f1a22ffd640 1 -- 172.21.15.52:0/4159038114 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a240072f0 con 0x7f1a34141010 2024-05-04T12:18:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.029+0000 7f1a38f42640 1 -- 172.21.15.52:0/4159038114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a34139ba0 con 0x7f1a34141010 2024-05-04T12:18:56.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.029+0000 7f1a38f42640 1 -- 172.21.15.52:0/4159038114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a34062e50 con 0x7f1a34141010 2024-05-04T12:18:56.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.029+0000 7f1a22ffd640 1 -- 172.21.15.52:0/4159038114 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1a2401a070 con 0x7f1a34141010 2024-05-04T12:18:56.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.029+0000 7f1a22ffd640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1a140678f0 0x7f1a14069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.029+0000 7f1a22ffd640 1 -- 172.21.15.52:0/4159038114 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1a24085030 con 0x7f1a34141010 2024-05-04T12:18:56.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.029+0000 7f1a31d9b640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1a140678f0 0x7f1a14069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.029+0000 7f1a31d9b640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1a140678f0 0x7f1a14069db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1a340750f0 tx=0x7f1a28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.033+0000 7f1a22ffd640 1 -- 172.21.15.52:0/4159038114 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1a24055b70 con 0x7f1a34141010 2024-05-04T12:18:56.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.189+0000 7f1a38f42640 1 -- 172.21.15.52:0/4159038114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a34075320 con 0x7f1a34141010 2024-05-04T12:18:56.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.189+0000 7f1a22ffd640 1 -- 172.21.15.52:0/4159038114 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f1a24055470 con 0x7f1a34141010 2024-05-04T12:18:56.193 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:18:56.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.193+0000 7f1a38f42640 1 -- 172.21.15.52:0/4159038114 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1a140678f0 msgr2=0x7f1a14069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.193+0000 7f1a38f42640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1a140678f0 0x7f1a14069db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1a340750f0 tx=0x7f1a28008040 comp rx=0 tx=0).stop 2024-05-04T12:18:56.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.193+0000 7f1a38f42640 1 -- 172.21.15.52:0/4159038114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a34141010 msgr2=0x7f1a3413a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.193+0000 7f1a38f42640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a34141010 0x7f1a3413a6f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400c3c0 tx=0x7f1a2400c880 comp rx=0 tx=0).stop 2024-05-04T12:18:56.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.193+0000 7f1a38f42640 1 -- 172.21.15.52:0/4159038114 shutdown_connections 2024-05-04T12:18:56.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.193+0000 7f1a38f42640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1a140678f0 0x7f1a14069db0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.193+0000 7f1a38f42640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34144570 0x7f1a340acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.193+0000 7f1a38f42640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a34143e80 0x7f1a3415f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.193+0000 7f1a38f42640 1 --2- 172.21.15.52:0/4159038114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a34141010 0x7f1a3413a6f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.193+0000 7f1a38f42640 1 -- 172.21.15.52:0/4159038114 >> 172.21.15.52:0/4159038114 conn(0x7f1a34072be0 msgr2=0x7f1a3415c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:56.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.193+0000 7f1a38f42640 1 -- 172.21.15.52:0/4159038114 shutdown_connections 2024-05-04T12:18:56.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.193+0000 7f1a38f42640 1 -- 172.21.15.52:0/4159038114 wait complete. 2024-05-04T12:18:56.211 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:18:56.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4ddd36d640 1 Processor -- start 2024-05-04T12:18:56.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4ddd36d640 1 -- start start 2024-05-04T12:18:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4ddd36d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4dd80c4280 0x7f4dd815b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4ddd36d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4dd815e970 0x7f4dd806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4ddd36d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dd815bad0 con 0x7f4dd80b2df0 2024-05-04T12:18:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4ddd36d640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f4dd80cd950 con 0x7f4dd80c4280 2024-05-04T12:18:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4ddd36d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f4dd815c370 con 0x7f4dd815e970 2024-05-04T12:18:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4dd67fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4dd80c4280 0x7f4dd815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4dd67fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4dd80c4280 0x7f4dd815b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37472/0 (socket says 172.21.15.52:37472) 2024-05-04T12:18:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4dd67fc640 1 -- 172.21.15.52:0/3891382424 learned_addr learned my addr 172.21.15.52:0/3891382424 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4dd77fe640 1 --2- 172.21.15.52:0/3891382424 >> v2:172.21.15.111:3300/0 conn(0x7f4dd815e970 0x7f4dd806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4dd67fc640 1 -- 172.21.15.52:0/3891382424 >> v2:172.21.15.111:3300/0 conn(0x7f4dd815e970 msgr2=0x7f4dd806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4dd67fc640 1 --2- 172.21.15.52:0/3891382424 >> v2:172.21.15.111:3300/0 conn(0x7f4dd815e970 0x7f4dd806e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4dd67fc640 1 -- 172.21.15.52:0/3891382424 >> v1:172.21.15.111:6789/0 conn(0x7f4dd80b2df0 legacy=0x7f4dd80c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.337+0000 7f4dd67fc640 1 -- 172.21.15.52:0/3891382424 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd815bdb0 con 0x7f4dd80c4280 2024-05-04T12:18:56.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.341+0000 7f4dd67fc640 1 --2- 172.21.15.52:0/3891382424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dd80c4280 0x7f4dd815b380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80d10c0 tx=0x7f4dcc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998e7db74d87a699 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:56.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.341+0000 7f4dd5ffb640 1 -- 172.21.15.52:0/3891382424 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0b2070 con 0x7f4dd80c4280 2024-05-04T12:18:56.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.341+0000 7f4dd5ffb640 1 -- 172.21.15.52:0/3891382424 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dcc0ad680 con 0x7f4dd80c4280 2024-05-04T12:18:56.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.341+0000 7f4dd5ffb640 1 -- 172.21.15.52:0/3891382424 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0ada50 con 0x7f4dd80c4280 2024-05-04T12:18:56.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.341+0000 7f4ddd36d640 1 -- 172.21.15.52:0/3891382424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dd80c4280 msgr2=0x7f4dd815b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.341+0000 7f4ddd36d640 1 --2- 172.21.15.52:0/3891382424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dd80c4280 0x7f4dd815b380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80d10c0 tx=0x7f4dcc01c490 comp rx=0 tx=0).stop 2024-05-04T12:18:56.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.341+0000 7f4ddd36d640 1 -- 172.21.15.52:0/3891382424 shutdown_connections 2024-05-04T12:18:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.341+0000 7f4ddd36d640 1 --2- 172.21.15.52:0/3891382424 >> v2:172.21.15.111:3300/0 conn(0x7f4dd815e970 0x7f4dd806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.341+0000 7f4ddd36d640 1 --2- 172.21.15.52:0/3891382424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dd80c4280 0x7f4dd815b380 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.341+0000 7f4ddd36d640 1 -- 172.21.15.52:0/3891382424 >> 172.21.15.52:0/3891382424 conn(0x7f4dd80d12c0 msgr2=0x7f4dd80d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.341+0000 7f4ddd36d640 1 -- 172.21.15.52:0/3891382424 shutdown_connections 2024-05-04T12:18:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.341+0000 7f4ddd36d640 1 -- 172.21.15.52:0/3891382424 wait complete. 2024-05-04T12:18:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.341+0000 7f4ddd36d640 1 Processor -- start 2024-05-04T12:18:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.341+0000 7f4ddd36d640 1 -- start start 2024-05-04T12:18:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4ddd36d640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd80b2df0 0x7f4dd814c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4ddd36d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd80c4280 0x7f4dd814eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4ddd36d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dd815e970 0x7f4dd8157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4ddd36d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd806f280 con 0x7f4dd815e970 2024-05-04T12:18:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4ddd36d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd806f100 con 0x7f4dd80c4280 2024-05-04T12:18:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4ddd36d640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4dd806f400 con 0x7f4dd80b2df0 2024-05-04T12:18:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dd77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dd815e970 0x7f4dd8157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dd77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dd815e970 0x7f4dd8157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37476/0 (socket says 172.21.15.52:37476) 2024-05-04T12:18:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dd77fe640 1 -- 172.21.15.52:0/4254964283 learned_addr learned my addr 172.21.15.52:0/4254964283 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dd67fc640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd80c4280 0x7f4dd814eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dd77fe640 1 -- 172.21.15.52:0/4254964283 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd80b2df0 msgr2=0x7f4dd814c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dd6ffd640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd80b2df0 0x7f4dd814c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dd77fe640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd80b2df0 0x7f4dd814c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dd77fe640 1 -- 172.21.15.52:0/4254964283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd80c4280 msgr2=0x7f4dd814eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dd77fe640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd80c4280 0x7f4dd814eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dd77fe640 1 -- 172.21.15.52:0/4254964283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd8159e40 con 0x7f4dd815e970 2024-05-04T12:18:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dd67fc640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd80c4280 0x7f4dd814eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dd6ffd640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd80b2df0 0x7f4dd814c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dd77fe640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dd815e970 0x7f4dd8157ca0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0007e60 tx=0x7f4dd000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dbf7fe640 1 -- 172.21.15.52:0/4254964283 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0014420 con 0x7f4dd815e970 2024-05-04T12:18:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dbf7fe640 1 -- 172.21.15.52:0/4254964283 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd0019070 con 0x7f4dd815e970 2024-05-04T12:18:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4ddd36d640 1 -- 172.21.15.52:0/4254964283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd815a0e0 con 0x7f4dd815e970 2024-05-04T12:18:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dbf7fe640 1 -- 172.21.15.52:0/4254964283 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0014820 con 0x7f4dd815e970 2024-05-04T12:18:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4ddd36d640 1 -- 172.21.15.52:0/4254964283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd814e350 con 0x7f4dd815e970 2024-05-04T12:18:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4ddd36d640 1 -- 172.21.15.52:0/4254964283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd815a600 con 0x7f4dd815e970 2024-05-04T12:18:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dbf7fe640 1 -- 172.21.15.52:0/4254964283 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4dd00149c0 con 0x7f4dd815e970 2024-05-04T12:18:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dbf7fe640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4dc40678c0 0x7f4dc4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dbf7fe640 1 -- 172.21.15.52:0/4254964283 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4dd00864e0 con 0x7f4dd815e970 2024-05-04T12:18:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.345+0000 7f4dd6ffd640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4dc40678c0 0x7f4dc4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.349+0000 7f4dd6ffd640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4dc40678c0 0x7f4dc4069d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4dd81574d0 tx=0x7f4dc8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:56.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.349+0000 7f4dbf7fe640 1 -- 172.21.15.52:0/4254964283 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4dd0057020 con 0x7f4dd815e970 2024-05-04T12:18:56.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.513+0000 7f4ddd36d640 1 -- 172.21.15.52:0/4254964283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4dd8062e50 con 0x7f4dd815e970 2024-05-04T12:18:56.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.513+0000 7f4dbf7fe640 1 -- 172.21.15.52:0/4254964283 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f4dd0056920 con 0x7f4dd815e970 2024-05-04T12:18:56.518 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:18:56.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.517+0000 7f4ddd36d640 1 -- 172.21.15.52:0/4254964283 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4dc40678c0 msgr2=0x7f4dc4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.517+0000 7f4ddd36d640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4dc40678c0 0x7f4dc4069d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4dd81574d0 tx=0x7f4dc8009040 comp rx=0 tx=0).stop 2024-05-04T12:18:56.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.517+0000 7f4ddd36d640 1 -- 172.21.15.52:0/4254964283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dd815e970 msgr2=0x7f4dd8157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.517+0000 7f4ddd36d640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dd815e970 0x7f4dd8157ca0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0007e60 tx=0x7f4dd000e4b0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.517+0000 7f4ddd36d640 1 -- 172.21.15.52:0/4254964283 shutdown_connections 2024-05-04T12:18:56.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.517+0000 7f4ddd36d640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4dc40678c0 0x7f4dc4069d80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.517+0000 7f4ddd36d640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dd815e970 0x7f4dd8157ca0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.517+0000 7f4ddd36d640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd80c4280 0x7f4dd814eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.517+0000 7f4ddd36d640 1 --2- 172.21.15.52:0/4254964283 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd80b2df0 0x7f4dd814c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.517+0000 7f4ddd36d640 1 -- 172.21.15.52:0/4254964283 >> 172.21.15.52:0/4254964283 conn(0x7f4dd80d12c0 msgr2=0x7f4dd81542c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:56.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.517+0000 7f4ddd36d640 1 -- 172.21.15.52:0/4254964283 shutdown_connections 2024-05-04T12:18:56.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.517+0000 7f4ddd36d640 1 -- 172.21.15.52:0/4254964283 wait complete. 2024-05-04T12:18:56.537 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:18:56.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f45823f7640 1 Processor -- start 2024-05-04T12:18:56.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f45823f7640 1 -- start start 2024-05-04T12:18:56.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f45823f7640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f457c0cf770 0x7f457c06f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f45823f7640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f457c0ccb80 con 0x7f457c15bd90 2024-05-04T12:18:56.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f45823f7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f457c0cce60 con 0x7f457c15c170 2024-05-04T12:18:56.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f45823f7640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f457c0cc930 con 0x7f457c0cf770 2024-05-04T12:18:56.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457bfff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f457c15bd90 0x7f457c0cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49836/0 (socket says 172.21.15.52:49836) 2024-05-04T12:18:56.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457bfff640 1 -- 172.21.15.52:0/3360124025 learned_addr learned my addr 172.21.15.52:0/3360124025 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:56.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 -- 172.21.15.52:0/3360124025 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3790236093 0 0) 0x7f457c0ccb80 con 0x7f457c15bd90 2024-05-04T12:18:56.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 -- 172.21.15.52:0/3360124025 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4568003530 con 0x7f457c15bd90 2024-05-04T12:18:56.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f45737fe640 1 --2- 172.21.15.52:0/3360124025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f457c0cf770 0x7f457c06f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 -- 172.21.15.52:0/3360124025 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2189862786 0 0) 0x7f4568003530 con 0x7f457c15bd90 2024-05-04T12:18:56.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 -- 172.21.15.52:0/3360124025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f457c0cf770 msgr2=0x7f457c06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:56.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 --2- 172.21.15.52:0/3360124025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f457c0cf770 0x7f457c06f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 -- 172.21.15.52:0/3360124025 >> v1:172.21.15.111:6789/0 conn(0x7f457c15c170 legacy=0x7f457c06cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 -- 172.21.15.52:0/3360124025 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457c0cd140 con 0x7f457c15bd90 2024-05-04T12:18:56.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 -- 172.21.15.52:0/3360124025 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f456c001680 con 0x7f457c15bd90 2024-05-04T12:18:56.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 -- 172.21.15.52:0/3360124025 >> v1:172.21.15.52:6789/0 conn(0x7f457c15bd90 legacy=0x7f457c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 --2- 172.21.15.52:0/3360124025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f457c0cf770 0x7f45680039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 --2- 172.21.15.52:0/3360124025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4568003f70 0x7f4568004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 --2- 172.21.15.52:0/3360124025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4568004d50 0x7f45680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 -- 172.21.15.52:0/3360124025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f457c0cc930 con 0x7f4568004d50 2024-05-04T12:18:56.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 -- 172.21.15.52:0/3360124025 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f456800bb60 con 0x7f4568003f70 2024-05-04T12:18:56.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f457b7fe640 1 -- 172.21.15.52:0/3360124025 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f456800bd60 con 0x7f457c0cf770 2024-05-04T12:18:56.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f4580994640 1 --2- 172.21.15.52:0/3360124025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4568004d50 0x7f45680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f45737fe640 1 --2- 172.21.15.52:0/3360124025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4568003f70 0x7f4568004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f4580994640 1 -- 172.21.15.52:0/3360124025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f457c0cf770 msgr2=0x7f45680039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:56.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f4580994640 1 --2- 172.21.15.52:0/3360124025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f457c0cf770 0x7f45680039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f4580994640 1 -- 172.21.15.52:0/3360124025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4568003f70 msgr2=0x7f4568004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:56.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f4580994640 1 --2- 172.21.15.52:0/3360124025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4568003f70 0x7f4568004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f4580994640 1 -- 172.21.15.52:0/3360124025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4568003530 con 0x7f4568004d50 2024-05-04T12:18:56.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.653+0000 7f45737fe640 1 --2- 172.21.15.52:0/3360124025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4568003f70 0x7f4568004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:56.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f4580994640 1 --2- 172.21.15.52:0/3360124025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4568004d50 0x7f45680051d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4574001a50 tx=0x7f457401e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e50471ef7c5756 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:56.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f457b7fe640 1 -- 172.21.15.52:0/3360124025 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574001a90 con 0x7f4568004d50 2024-05-04T12:18:56.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f457b7fe640 1 -- 172.21.15.52:0/3360124025 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4574003b80 con 0x7f4568004d50 2024-05-04T12:18:56.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f457b7fe640 1 -- 172.21.15.52:0/3360124025 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45740050d0 con 0x7f4568004d50 2024-05-04T12:18:56.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f45823f7640 1 -- 172.21.15.52:0/3360124025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4568004d50 msgr2=0x7f45680051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f45823f7640 1 --2- 172.21.15.52:0/3360124025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4568004d50 0x7f45680051d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4574001a50 tx=0x7f457401e9b0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f4580994640 1 -- 172.21.15.52:0/3360124025 reap_dead start 2024-05-04T12:18:56.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f45823f7640 1 -- 172.21.15.52:0/3360124025 shutdown_connections 2024-05-04T12:18:56.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f45823f7640 1 -- 172.21.15.52:0/3360124025 >> 172.21.15.52:0/3360124025 conn(0x7f457c067730 msgr2=0x7f457c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:56.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f45823f7640 1 -- 172.21.15.52:0/3360124025 shutdown_connections 2024-05-04T12:18:56.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f45823f7640 1 -- 172.21.15.52:0/3360124025 wait complete. 2024-05-04T12:18:56.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f45823f7640 1 Processor -- start 2024-05-04T12:18:56.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f45823f7640 1 -- start start 2024-05-04T12:18:56.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f45823f7640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4568004d50 0x7f457c15a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f45823f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f457c0b2f50 0x7f457c0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f45823f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f457c14e680 0x7f457c14eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f45823f7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f457c06d350 con 0x7f457c14e680 2024-05-04T12:18:56.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f45823f7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f457c0cd4b0 con 0x7f457c0b2f50 2024-05-04T12:18:56.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f45823f7640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f457c06d4d0 con 0x7f4568004d50 2024-05-04T12:18:56.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f4580994640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f457c14e680 0x7f457c14eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f4580994640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f457c14e680 0x7f457c14eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37484/0 (socket says 172.21.15.52:37484) 2024-05-04T12:18:56.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.657+0000 7f4580994640 1 -- 172.21.15.52:0/458999591 learned_addr learned my addr 172.21.15.52:0/458999591 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f45737fe640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f457c0b2f50 0x7f457c0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f457bfff640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4568004d50 0x7f457c15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f4580994640 1 -- 172.21.15.52:0/458999591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4568004d50 msgr2=0x7f457c15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f4580994640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4568004d50 0x7f457c15a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f4580994640 1 -- 172.21.15.52:0/458999591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f457c0b2f50 msgr2=0x7f457c0adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f4580994640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f457c0b2f50 0x7f457c0adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f4580994640 1 -- 172.21.15.52:0/458999591 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457c0b33d0 con 0x7f457c14e680 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f45737fe640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f457c0b2f50 0x7f457c0adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f4580994640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f457c14e680 0x7f457c14eb00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f457409a3d0 tx=0x7f4568005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f45797fa640 1 -- 172.21.15.52:0/458999591 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574007dc0 con 0x7f457c14e680 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f45797fa640 1 -- 172.21.15.52:0/458999591 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4574003980 con 0x7f457c14e680 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f45823f7640 1 -- 172.21.15.52:0/458999591 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457c0c3d10 con 0x7f457c14e680 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f45797fa640 1 -- 172.21.15.52:0/458999591 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45740050d0 con 0x7f457c14e680 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f45823f7640 1 -- 172.21.15.52:0/458999591 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f457c0c42d0 con 0x7f457c14e680 2024-05-04T12:18:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f45823f7640 1 -- 172.21.15.52:0/458999591 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f457c062e50 con 0x7f457c14e680 2024-05-04T12:18:56.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f45797fa640 1 -- 172.21.15.52:0/458999591 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4574005270 con 0x7f457c14e680 2024-05-04T12:18:56.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f45797fa640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f455c0678c0 0x7f455c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f45797fa640 1 -- 172.21.15.52:0/458999591 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f457411f5b0 con 0x7f457c14e680 2024-05-04T12:18:56.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f457bfff640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f455c0678c0 0x7f455c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.661+0000 7f457bfff640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f455c0678c0 0x7f455c069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f456c001870 tx=0x7f456c00c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:56.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.665+0000 7f45797fa640 1 -- 172.21.15.52:0/458999591 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f45740186c0 con 0x7f457c14e680 2024-05-04T12:18:56.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.821+0000 7f45823f7640 1 -- 172.21.15.52:0/458999591 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f457c075320 con 0x7f457c14e680 2024-05-04T12:18:56.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.821+0000 7f45797fa640 1 -- 172.21.15.52:0/458999591 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f45740f0200 con 0x7f457c14e680 2024-05-04T12:18:56.824 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:18:56.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.821+0000 7f45823f7640 1 -- 172.21.15.52:0/458999591 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f455c0678c0 msgr2=0x7f455c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.821+0000 7f45823f7640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f455c0678c0 0x7f455c069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f456c001870 tx=0x7f456c00c040 comp rx=0 tx=0).stop 2024-05-04T12:18:56.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.821+0000 7f45823f7640 1 -- 172.21.15.52:0/458999591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f457c14e680 msgr2=0x7f457c14eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.821+0000 7f45823f7640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f457c14e680 0x7f457c14eb00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f457409a3d0 tx=0x7f4568005840 comp rx=0 tx=0).stop 2024-05-04T12:18:56.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.821+0000 7f45823f7640 1 -- 172.21.15.52:0/458999591 shutdown_connections 2024-05-04T12:18:56.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.821+0000 7f45823f7640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f455c0678c0 0x7f455c069d80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.821+0000 7f45823f7640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f457c14e680 0x7f457c14eb00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.821+0000 7f45823f7640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f457c0b2f50 0x7f457c0adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.821+0000 7f45823f7640 1 --2- 172.21.15.52:0/458999591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4568004d50 0x7f457c15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.821+0000 7f45823f7640 1 -- 172.21.15.52:0/458999591 >> 172.21.15.52:0/458999591 conn(0x7f457c067730 msgr2=0x7f457c06f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:56.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.821+0000 7f45823f7640 1 -- 172.21.15.52:0/458999591 shutdown_connections 2024-05-04T12:18:56.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.821+0000 7f45823f7640 1 -- 172.21.15.52:0/458999591 wait complete. 2024-05-04T12:18:56.842 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-04T12:18:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1333d99640 1 Processor -- start 2024-05-04T12:18:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1333d99640 1 -- start start 2024-05-04T12:18:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1333d99640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f132c162650 0x7f132c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1333d99640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f132c15e7f0 con 0x7f132c15f570 2024-05-04T12:18:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1333d99640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f132c06df60 con 0x7f132c0ce490 2024-05-04T12:18:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1333d99640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f132c15ea90 con 0x7f132c162650 2024-05-04T12:18:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1331334640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f132c15f570 0x7f132c0cdd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49844/0 (socket says 172.21.15.52:49844) 2024-05-04T12:18:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1331334640 1 -- 172.21.15.52:0/3384718816 learned_addr learned my addr 172.21.15.52:0/3384718816 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1331b35640 1 --2- 172.21.15.52:0/3384718816 >> v2:172.21.15.111:3300/0 conn(0x7f132c162650 0x7f132c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 -- 172.21.15.52:0/3384718816 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3392361771 0 0) 0x7f132c15e7f0 con 0x7f132c15f570 2024-05-04T12:18:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 -- 172.21.15.52:0/3384718816 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1314003530 con 0x7f132c15f570 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 -- 172.21.15.52:0/3384718816 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2382548380 0 0) 0x7f1314003530 con 0x7f132c15f570 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 -- 172.21.15.52:0/3384718816 >> v2:172.21.15.111:3300/0 conn(0x7f132c162650 msgr2=0x7f132c06d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 --2- 172.21.15.52:0/3384718816 >> v2:172.21.15.111:3300/0 conn(0x7f132c162650 0x7f132c06d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 -- 172.21.15.52:0/3384718816 >> v1:172.21.15.111:6789/0 conn(0x7f132c0ce490 legacy=0x7f132c0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 -- 172.21.15.52:0/3384718816 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c15ed70 con 0x7f132c15f570 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 -- 172.21.15.52:0/3384718816 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f1320001680 con 0x7f132c15f570 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 -- 172.21.15.52:0/3384718816 >> v1:172.21.15.52:6789/0 conn(0x7f132c15f570 legacy=0x7f132c0cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 --2- 172.21.15.52:0/3384718816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13140039c0 0x7f1314003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 --2- 172.21.15.52:0/3384718816 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1314004810 0x7f1314004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 --2- 172.21.15.52:0/3384718816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13140051d0 0x7f131400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 -- 172.21.15.52:0/3384718816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f132c15ea90 con 0x7f13140039c0 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 -- 172.21.15.52:0/3384718816 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f131400c070 con 0x7f13140051d0 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1330b33640 1 -- 172.21.15.52:0/3384718816 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f131400c270 con 0x7f1314004810 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1331b35640 1 --2- 172.21.15.52:0/3384718816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13140039c0 0x7f1314003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1331334640 1 --2- 172.21.15.52:0/3384718816 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1314004810 0x7f1314004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1331b35640 1 -- 172.21.15.52:0/3384718816 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1314004810 msgr2=0x7f1314004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1332336640 1 --2- 172.21.15.52:0/3384718816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13140051d0 0x7f131400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1331b35640 1 --2- 172.21.15.52:0/3384718816 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1314004810 0x7f1314004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1331b35640 1 -- 172.21.15.52:0/3384718816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13140051d0 msgr2=0x7f131400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1331b35640 1 --2- 172.21.15.52:0/3384718816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13140051d0 0x7f131400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1331b35640 1 -- 172.21.15.52:0/3384718816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1314003530 con 0x7f13140039c0 2024-05-04T12:18:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1332336640 1 --2- 172.21.15.52:0/3384718816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13140051d0 0x7f131400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.961+0000 7f1332336640 1 -- 172.21.15.52:0/3384718816 reap_dead start 2024-05-04T12:18:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.965+0000 7f1331b35640 1 --2- 172.21.15.52:0/3384718816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13140039c0 0x7f1314003de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f131c009fc0 tx=0x7f131c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd94046b0b0f193a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.965+0000 7f1330b33640 1 -- 172.21.15.52:0/3384718816 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c0b2070 con 0x7f13140039c0 2024-05-04T12:18:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.965+0000 7f1330b33640 1 -- 172.21.15.52:0/3384718816 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f131c0a5b80 con 0x7f13140039c0 2024-05-04T12:18:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.965+0000 7f1330b33640 1 -- 172.21.15.52:0/3384718816 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c009c10 con 0x7f13140039c0 2024-05-04T12:18:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.965+0000 7f1333d99640 1 -- 172.21.15.52:0/3384718816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13140039c0 msgr2=0x7f1314003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.965+0000 7f1333d99640 1 --2- 172.21.15.52:0/3384718816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13140039c0 0x7f1314003de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f131c009fc0 tx=0x7f131c01cc80 comp rx=0 tx=0).stop 2024-05-04T12:18:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.965+0000 7f1333d99640 1 -- 172.21.15.52:0/3384718816 shutdown_connections 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.965+0000 7f1333d99640 1 --2- 172.21.15.52:0/3384718816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13140039c0 0x7f1314003de0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.965+0000 7f1333d99640 1 -- 172.21.15.52:0/3384718816 >> 172.21.15.52:0/3384718816 conn(0x7f132c0b4000 msgr2=0x7f132c0cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.965+0000 7f1333d99640 1 -- 172.21.15.52:0/3384718816 shutdown_connections 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.965+0000 7f1333d99640 1 -- 172.21.15.52:0/3384718816 wait complete. 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.965+0000 7f1333d99640 1 Processor -- start 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.965+0000 7f1333d99640 1 -- start start 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1333d99640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13140039c0 0x7f132c1528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1333d99640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f132c14e740 0x7f132c154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1333d99640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f132c14ebc0 0x7f132c151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1333d99640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f132c0cfc00 con 0x7f132c14ebc0 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1333d99640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f132c0cfa80 con 0x7f132c14e740 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1333d99640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f132c0cfd80 con 0x7f13140039c0 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1332336640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f132c14ebc0 0x7f132c151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1332336640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f132c14ebc0 0x7f132c151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37502/0 (socket says 172.21.15.52:37502) 2024-05-04T12:18:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1332336640 1 -- 172.21.15.52:0/2558969093 learned_addr learned my addr 172.21.15.52:0/2558969093 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1331b35640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13140039c0 0x7f132c1528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1331334640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f132c14e740 0x7f132c154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1332336640 1 -- 172.21.15.52:0/2558969093 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13140039c0 msgr2=0x7f132c1528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1332336640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13140039c0 0x7f132c1528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1332336640 1 -- 172.21.15.52:0/2558969093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f132c14e740 msgr2=0x7f132c154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1332336640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f132c14e740 0x7f132c154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1332336640 1 -- 172.21.15.52:0/2558969093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c14f3d0 con 0x7f132c14ebc0 2024-05-04T12:18:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1331b35640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13140039c0 0x7f132c1528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1332336640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f132c14ebc0 0x7f132c151b60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f132c0685f0 tx=0x7f132400d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f132a7fc640 1 -- 172.21.15.52:0/2558969093 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324018070 con 0x7f132c14ebc0 2024-05-04T12:18:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f132a7fc640 1 -- 172.21.15.52:0/2558969093 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13240052f0 con 0x7f132c14ebc0 2024-05-04T12:18:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f132a7fc640 1 -- 172.21.15.52:0/2558969093 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13240054a0 con 0x7f132c14ebc0 2024-05-04T12:18:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1333d99640 1 -- 172.21.15.52:0/2558969093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132c14f6b0 con 0x7f132c14ebc0 2024-05-04T12:18:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1333d99640 1 -- 172.21.15.52:0/2558969093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132c152df0 con 0x7f132c14ebc0 2024-05-04T12:18:56.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1333d99640 1 -- 172.21.15.52:0/2558969093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f132c062e50 con 0x7f132c14ebc0 2024-05-04T12:18:56.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f132a7fc640 1 -- 172.21.15.52:0/2558969093 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f132401b070 con 0x7f132c14ebc0 2024-05-04T12:18:56.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f132a7fc640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13180678c0 0x7f1318069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:56.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f132a7fc640 1 -- 172.21.15.52:0/2558969093 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1324085140 con 0x7f132c14ebc0 2024-05-04T12:18:56.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1331b35640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13180678c0 0x7f1318069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:56.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.969+0000 7f1331b35640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13180678c0 0x7f1318069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f131c0af000 tx=0x7f131c0a7040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:56.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:56.973+0000 7f132a7fc640 1 -- 172.21.15.52:0/2558969093 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1324055c80 con 0x7f132c14ebc0 2024-05-04T12:18:57.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.141+0000 7f1333d99640 1 -- 172.21.15.52:0/2558969093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f132c075320 con 0x7f132c14ebc0 2024-05-04T12:18:57.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.141+0000 7f132a7fc640 1 -- 172.21.15.52:0/2558969093 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f1324055580 con 0x7f132c14ebc0 2024-05-04T12:18:57.146 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-04T12:18:57.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.145+0000 7f1333d99640 1 -- 172.21.15.52:0/2558969093 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13180678c0 msgr2=0x7f1318069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.145+0000 7f1333d99640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13180678c0 0x7f1318069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f131c0af000 tx=0x7f131c0a7040 comp rx=0 tx=0).stop 2024-05-04T12:18:57.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.145+0000 7f1333d99640 1 -- 172.21.15.52:0/2558969093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f132c14ebc0 msgr2=0x7f132c151b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.145+0000 7f1333d99640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f132c14ebc0 0x7f132c151b60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f132c0685f0 tx=0x7f132400d5e0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.145+0000 7f1333d99640 1 -- 172.21.15.52:0/2558969093 shutdown_connections 2024-05-04T12:18:57.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.145+0000 7f1333d99640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13180678c0 0x7f1318069d80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.145+0000 7f1333d99640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f132c14ebc0 0x7f132c151b60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.145+0000 7f1333d99640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f132c14e740 0x7f132c154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.145+0000 7f1333d99640 1 --2- 172.21.15.52:0/2558969093 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13140039c0 0x7f132c1528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.145+0000 7f1333d99640 1 -- 172.21.15.52:0/2558969093 >> 172.21.15.52:0/2558969093 conn(0x7f132c0b4000 msgr2=0x7f132c160510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:57.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.145+0000 7f1333d99640 1 -- 172.21.15.52:0/2558969093 shutdown_connections 2024-05-04T12:18:57.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.145+0000 7f1333d99640 1 -- 172.21.15.52:0/2558969093 wait complete. 2024-05-04T12:18:57.163 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:18:57.163 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T12:18:57.169 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:57.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.329+0000 7f7c0b354640 1 Processor -- start 2024-05-04T12:18:57.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.329+0000 7f7c0b354640 1 -- start start 2024-05-04T12:18:57.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.329+0000 7f7c0b354640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7c0413f000 0x7f7c0406b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.329+0000 7f7c0b354640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c0413f3e0 con 0x7f7c0406b750 2024-05-04T12:18:57.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.329+0000 7f7c0b354640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c0415c560 con 0x7f7c04160260 2024-05-04T12:18:57.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.329+0000 7f7c0b354640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f7c04160670 con 0x7f7c0413f000 2024-05-04T12:18:57.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.329+0000 7f7c090f0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7c0413f000 0x7f7c0406b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.329+0000 7f7c088ef640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f7c0406b750 0x7f7c0415fb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49848/0 (socket says 172.21.15.52:49848) 2024-05-04T12:18:57.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.329+0000 7f7c088ef640 1 -- 172.21.15.52:0/1578944823 learned_addr learned my addr 172.21.15.52:0/1578944823 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:57.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.329+0000 7f7c090f0640 1 -- 172.21.15.52:0/1578944823 >> v1:172.21.15.111:6789/0 conn(0x7f7c04160260 legacy=0x7f7c040a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:57.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.329+0000 7f7c090f0640 1 -- 172.21.15.52:0/1578944823 >> v1:172.21.15.52:6789/0 conn(0x7f7c0406b750 legacy=0x7f7c0415fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.329+0000 7f7c090f0640 1 -- 172.21.15.52:0/1578944823 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c041608c0 con 0x7f7c0413f000 2024-05-04T12:18:57.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7c090f0640 1 --2- 172.21.15.52:0/1578944823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c0413f000 0x7f7c0406b210 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7c000097c0 tx=0x7f7c0001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193449540aa389c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:57.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7bfbfff640 1 -- 172.21.15.52:0/1578944823 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c000b0070 con 0x7f7c0413f000 2024-05-04T12:18:57.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7bfbfff640 1 -- 172.21.15.52:0/1578944823 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c000ab400 con 0x7f7c0413f000 2024-05-04T12:18:57.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7bfbfff640 1 -- 172.21.15.52:0/1578944823 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c000ab7d0 con 0x7f7c0413f000 2024-05-04T12:18:57.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7c0b354640 1 -- 172.21.15.52:0/1578944823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c0413f000 msgr2=0x7f7c0406b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7c0b354640 1 --2- 172.21.15.52:0/1578944823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c0413f000 0x7f7c0406b210 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7c000097c0 tx=0x7f7c0001c3d0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7c0b354640 1 -- 172.21.15.52:0/1578944823 shutdown_connections 2024-05-04T12:18:57.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7c0b354640 1 --2- 172.21.15.52:0/1578944823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c0413f000 0x7f7c0406b210 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7c0b354640 1 -- 172.21.15.52:0/1578944823 >> 172.21.15.52:0/1578944823 conn(0x7f7c04070690 msgr2=0x7f7c04070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:57.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7c0b354640 1 -- 172.21.15.52:0/1578944823 shutdown_connections 2024-05-04T12:18:57.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7c0b354640 1 -- 172.21.15.52:0/1578944823 wait complete. 2024-05-04T12:18:57.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7c0b354640 1 Processor -- start 2024-05-04T12:18:57.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7c0b354640 1 -- start start 2024-05-04T12:18:57.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7c0b354640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c0406b750 0x7f7c04138d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7c0b354640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c0413f000 0x7f7c04152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.333+0000 7f7c0b354640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c04160260 0x7f7c041538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c0b354640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0415d480 con 0x7f7c04160260 2024-05-04T12:18:57.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c0b354640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0415d300 con 0x7f7c0406b750 2024-05-04T12:18:57.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c0b354640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f7c04169cf0 con 0x7f7c0413f000 2024-05-04T12:18:57.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c098f1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c04160260 0x7f7c041538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c098f1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c04160260 0x7f7c041538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37518/0 (socket says 172.21.15.52:37518) 2024-05-04T12:18:57.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c098f1640 1 -- 172.21.15.52:0/381098425 learned_addr learned my addr 172.21.15.52:0/381098425 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:57.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c088ef640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c0413f000 0x7f7c04152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c098f1640 1 -- 172.21.15.52:0/381098425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c0413f000 msgr2=0x7f7c04152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c090f0640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c0406b750 0x7f7c04138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c098f1640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c0413f000 0x7f7c04152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c098f1640 1 -- 172.21.15.52:0/381098425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c0406b750 msgr2=0x7f7c04138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c098f1640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c0406b750 0x7f7c04138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c098f1640 1 -- 172.21.15.52:0/381098425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c041534f0 con 0x7f7c04160260 2024-05-04T12:18:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c088ef640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c0413f000 0x7f7c04152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c098f1640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c04160260 0x7f7c041538c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00c780 tx=0x7f7bfc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7bf9ffb640 1 -- 172.21.15.52:0/381098425 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc004b70 con 0x7f7c04160260 2024-05-04T12:18:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7bf9ffb640 1 -- 172.21.15.52:0/381098425 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bfc004d10 con 0x7f7c04160260 2024-05-04T12:18:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c0b354640 1 -- 172.21.15.52:0/381098425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c04150f00 con 0x7f7c04160260 2024-05-04T12:18:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7bf9ffb640 1 -- 172.21.15.52:0/381098425 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc00aa90 con 0x7f7c04160260 2024-05-04T12:18:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c0b354640 1 -- 172.21.15.52:0/381098425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c041514c0 con 0x7f7c04160260 2024-05-04T12:18:57.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c0b354640 1 -- 172.21.15.52:0/381098425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c04062e50 con 0x7f7c04160260 2024-05-04T12:18:57.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7bf9ffb640 1 -- 172.21.15.52:0/381098425 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7bfc00ac30 con 0x7f7c04160260 2024-05-04T12:18:57.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7bf9ffb640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7be80678f0 0x7f7be8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7bf9ffb640 1 -- 172.21.15.52:0/381098425 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7bfc085e70 con 0x7f7c04160260 2024-05-04T12:18:57.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c090f0640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7be80678f0 0x7f7be8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.337+0000 7f7c090f0640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7be80678f0 0x7f7be8069db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7c000ad000 tx=0x7f7c00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:57.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.341+0000 7f7bf9ffb640 1 -- 172.21.15.52:0/381098425 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7bfc020070 con 0x7f7c04160260 2024-05-04T12:18:57.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.509+0000 7f7c0b354640 1 -- 172.21.15.52:0/381098425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c04075320 con 0x7f7c04160260 2024-05-04T12:18:57.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.509+0000 7f7bf9ffb640 1 -- 172.21.15.52:0/381098425 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f7bfc056a40 con 0x7f7c04160260 2024-05-04T12:18:57.514 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:57.514 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:57.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.513+0000 7f7c0b354640 1 -- 172.21.15.52:0/381098425 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7be80678f0 msgr2=0x7f7be8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.513+0000 7f7c0b354640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7be80678f0 0x7f7be8069db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7c000ad000 tx=0x7f7c00008040 comp rx=0 tx=0).stop 2024-05-04T12:18:57.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.513+0000 7f7c0b354640 1 -- 172.21.15.52:0/381098425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c04160260 msgr2=0x7f7c041538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.513+0000 7f7c0b354640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c04160260 0x7f7c041538c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00c780 tx=0x7f7bfc00cc40 comp rx=0 tx=0).stop 2024-05-04T12:18:57.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.513+0000 7f7c0b354640 1 -- 172.21.15.52:0/381098425 shutdown_connections 2024-05-04T12:18:57.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.513+0000 7f7c0b354640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f7be80678f0 0x7f7be8069db0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.513+0000 7f7c0b354640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c04160260 0x7f7c041538c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.513+0000 7f7c0b354640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c0413f000 0x7f7c04152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.513+0000 7f7c0b354640 1 --2- 172.21.15.52:0/381098425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c0406b750 0x7f7c04138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.513+0000 7f7c0b354640 1 -- 172.21.15.52:0/381098425 >> 172.21.15.52:0/381098425 conn(0x7f7c04070690 msgr2=0x7f7c0406ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:57.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.513+0000 7f7c0b354640 1 -- 172.21.15.52:0/381098425 shutdown_connections 2024-05-04T12:18:57.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.513+0000 7f7c0b354640 1 -- 172.21.15.52:0/381098425 wait complete. 2024-05-04T12:18:57.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:18:57.534 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:18:57.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f0064a9e640 1 Processor -- start 2024-05-04T12:18:57.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f0064a9e640 1 -- start start 2024-05-04T12:18:57.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f0064a9e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00600c4630 0x7f00600706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f0064a9e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f00600c5480 0x7f006015fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f0064a9e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f006015c6c0 con 0x7f0060070c10 2024-05-04T12:18:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f0064a9e640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f0060071070 con 0x7f00600c5480 2024-05-04T12:18:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f0064a9e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f00600c5a80 con 0x7f00600c4630 2024-05-04T12:18:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f005ed9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f00600c5480 0x7f006015fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f005ed9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f00600c5480 0x7f006015fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37534/0 (socket says 172.21.15.52:37534) 2024-05-04T12:18:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f005ed9d640 1 -- 172.21.15.52:0/1149225735 learned_addr learned my addr 172.21.15.52:0/1149225735 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f005e59c640 1 --2- 172.21.15.52:0/1149225735 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00600c4630 0x7f00600706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f005ed9d640 1 -- 172.21.15.52:0/1149225735 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00600c4630 msgr2=0x7f00600706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f005ed9d640 1 --2- 172.21.15.52:0/1149225735 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00600c4630 0x7f00600706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f005ed9d640 1 -- 172.21.15.52:0/1149225735 >> v1:172.21.15.111:6789/0 conn(0x7f0060070c10 legacy=0x7f00600c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.657+0000 7f005ed9d640 1 -- 172.21.15.52:0/1149225735 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006015c9a0 con 0x7f00600c5480 2024-05-04T12:18:57.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f005ed9d640 1 --2- 172.21.15.52:0/1149225735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00600c5480 0x7f006015fa60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0058009e80 tx=0x7f005801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166b60b7ed8875d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:57.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f005dd9b640 1 -- 172.21.15.52:0/1149225735 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00580a5760 con 0x7f00600c5480 2024-05-04T12:18:57.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f005dd9b640 1 -- 172.21.15.52:0/1149225735 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00580a5900 con 0x7f00600c5480 2024-05-04T12:18:57.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f005dd9b640 1 -- 172.21.15.52:0/1149225735 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00580a5cd0 con 0x7f00600c5480 2024-05-04T12:18:57.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 -- 172.21.15.52:0/1149225735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00600c5480 msgr2=0x7f006015fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 --2- 172.21.15.52:0/1149225735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00600c5480 0x7f006015fa60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0058009e80 tx=0x7f005801c9d0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 -- 172.21.15.52:0/1149225735 shutdown_connections 2024-05-04T12:18:57.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 --2- 172.21.15.52:0/1149225735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00600c5480 0x7f006015fa60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 --2- 172.21.15.52:0/1149225735 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00600c4630 0x7f00600706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 -- 172.21.15.52:0/1149225735 >> 172.21.15.52:0/1149225735 conn(0x7f00600d12c0 msgr2=0x7f00600d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:57.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 -- 172.21.15.52:0/1149225735 shutdown_connections 2024-05-04T12:18:57.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 -- 172.21.15.52:0/1149225735 wait complete. 2024-05-04T12:18:57.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 Processor -- start 2024-05-04T12:18:57.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 -- start start 2024-05-04T12:18:57.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0060070c10 0x7f006014af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00600c4630 0x7f00601559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00600c5480 0x7f006015af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f006015cff0 con 0x7f00600c5480 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0060168e00 con 0x7f0060070c10 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0064a9e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f0060168f80 con 0x7f00600c4630 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f005ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00600c5480 0x7f006015af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f005ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00600c5480 0x7f006015af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37540/0 (socket says 172.21.15.52:37540) 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f005ed9d640 1 -- 172.21.15.52:0/506520405 learned_addr learned my addr 172.21.15.52:0/506520405 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f005e59c640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0060070c10 0x7f006014af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0055d9b640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00600c4630 0x7f00601559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f005ed9d640 1 -- 172.21.15.52:0/506520405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00600c4630 msgr2=0x7f00601559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f005ed9d640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00600c4630 0x7f00601559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f005ed9d640 1 -- 172.21.15.52:0/506520405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0060070c10 msgr2=0x7f006014af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f005ed9d640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0060070c10 0x7f006014af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f005ed9d640 1 -- 172.21.15.52:0/506520405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006014d890 con 0x7f00600c5480 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.661+0000 7f0055d9b640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00600c4630 0x7f00601559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.665+0000 7f005ed9d640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00600c5480 0x7f006015af40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f00580af040 tx=0x7f0058019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.665+0000 7f00577fe640 1 -- 172.21.15.52:0/506520405 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00580b8470 con 0x7f00600c5480 2024-05-04T12:18:57.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.665+0000 7f00577fe640 1 -- 172.21.15.52:0/506520405 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00580b8650 con 0x7f00600c5480 2024-05-04T12:18:57.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.665+0000 7f0064a9e640 1 -- 172.21.15.52:0/506520405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00601fb080 con 0x7f00600c5480 2024-05-04T12:18:57.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.665+0000 7f00577fe640 1 -- 172.21.15.52:0/506520405 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00580b8a40 con 0x7f00600c5480 2024-05-04T12:18:57.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.665+0000 7f0064a9e640 1 -- 172.21.15.52:0/506520405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00601fb640 con 0x7f00600c5480 2024-05-04T12:18:57.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.665+0000 7f0064a9e640 1 -- 172.21.15.52:0/506520405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0060062e50 con 0x7f00600c5480 2024-05-04T12:18:57.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.665+0000 7f00577fe640 1 -- 172.21.15.52:0/506520405 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f00580c0030 con 0x7f00600c5480 2024-05-04T12:18:57.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.665+0000 7f00577fe640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f00400678c0 0x7f0040069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.665+0000 7f00577fe640 1 -- 172.21.15.52:0/506520405 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0058120720 con 0x7f00600c5480 2024-05-04T12:18:57.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.665+0000 7f005e59c640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f00400678c0 0x7f0040069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.665+0000 7f005e59c640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f00400678c0 0x7f0040069d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0050009790 tx=0x7f0050008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:57.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.669+0000 7f00577fe640 1 -- 172.21.15.52:0/506520405 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f00580c7020 con 0x7f00600c5480 2024-05-04T12:18:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.829+0000 7f0064a9e640 1 -- 172.21.15.52:0/506520405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0060075350 con 0x7f00600c5480 2024-05-04T12:18:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.829+0000 7f00577fe640 1 -- 172.21.15.52:0/506520405 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f00580165c0 con 0x7f00600c5480 2024-05-04T12:18:57.834 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:18:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.833+0000 7f0064a9e640 1 -- 172.21.15.52:0/506520405 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f00400678c0 msgr2=0x7f0040069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.833+0000 7f0064a9e640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f00400678c0 0x7f0040069d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0050009790 tx=0x7f0050008040 comp rx=0 tx=0).stop 2024-05-04T12:18:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.833+0000 7f0064a9e640 1 -- 172.21.15.52:0/506520405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00600c5480 msgr2=0x7f006015af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.833+0000 7f0064a9e640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00600c5480 0x7f006015af40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f00580af040 tx=0x7f0058019770 comp rx=0 tx=0).stop 2024-05-04T12:18:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.833+0000 7f0064a9e640 1 -- 172.21.15.52:0/506520405 shutdown_connections 2024-05-04T12:18:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.833+0000 7f0064a9e640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f00400678c0 0x7f0040069d80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.833+0000 7f0064a9e640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00600c5480 0x7f006015af40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.833+0000 7f0064a9e640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00600c4630 0x7f00601559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.833+0000 7f0064a9e640 1 --2- 172.21.15.52:0/506520405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0060070c10 0x7f006014af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.833+0000 7f0064a9e640 1 -- 172.21.15.52:0/506520405 >> 172.21.15.52:0/506520405 conn(0x7f00600d12c0 msgr2=0x7f00601496f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:57.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.833+0000 7f0064a9e640 1 -- 172.21.15.52:0/506520405 shutdown_connections 2024-05-04T12:18:57.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.833+0000 7f0064a9e640 1 -- 172.21.15.52:0/506520405 wait complete. 2024-05-04T12:18:57.852 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:18:57.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa64eb3a640 1 Processor -- start 2024-05-04T12:18:57.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa64eb3a640 1 -- start start 2024-05-04T12:18:57.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa64eb3a640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa648141010 0x7fa6480d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa64eb3a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa648144570 0x7fa648143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa64eb3a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6480d29c0 con 0x7fa6480d2540 2024-05-04T12:18:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa64eb3a640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fa6480d2ca0 con 0x7fa648144570 2024-05-04T12:18:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa64eb3a640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa6480d2f80 con 0x7fa648141010 2024-05-04T12:18:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa647fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa648144570 0x7fa648143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa647fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa648144570 0x7fa648143940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37554/0 (socket says 172.21.15.52:37554) 2024-05-04T12:18:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa647fff640 1 -- 172.21.15.52:0/985735175 learned_addr learned my addr 172.21.15.52:0/985735175 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa64c8d6640 1 --2- 172.21.15.52:0/985735175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa648141010 0x7fa6480d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa647fff640 1 -- 172.21.15.52:0/985735175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa648141010 msgr2=0x7fa6480d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa647fff640 1 --2- 172.21.15.52:0/985735175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa648141010 0x7fa6480d2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa647fff640 1 -- 172.21.15.52:0/985735175 >> v1:172.21.15.111:6789/0 conn(0x7fa6480d2540 legacy=0x7fa648161db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.977+0000 7fa647fff640 1 -- 172.21.15.52:0/985735175 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6480d3260 con 0x7fa648144570 2024-05-04T12:18:57.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.981+0000 7fa647fff640 1 --2- 172.21.15.52:0/985735175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa648144570 0x7fa648143940 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009870 tx=0x7fa63c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f45c6839ec30b04 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:57.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.981+0000 7fa6477fe640 1 -- 172.21.15.52:0/985735175 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c0b0070 con 0x7fa648144570 2024-05-04T12:18:57.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.981+0000 7fa6477fe640 1 -- 172.21.15.52:0/985735175 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa63c0ab600 con 0x7fa648144570 2024-05-04T12:18:57.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.981+0000 7fa6477fe640 1 -- 172.21.15.52:0/985735175 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c0ab9d0 con 0x7fa648144570 2024-05-04T12:18:57.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.981+0000 7fa64eb3a640 1 -- 172.21.15.52:0/985735175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa648144570 msgr2=0x7fa648143940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.981+0000 7fa64eb3a640 1 --2- 172.21.15.52:0/985735175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa648144570 0x7fa648143940 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009870 tx=0x7fa63c01c460 comp rx=0 tx=0).stop 2024-05-04T12:18:57.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.981+0000 7fa64eb3a640 1 -- 172.21.15.52:0/985735175 shutdown_connections 2024-05-04T12:18:57.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.981+0000 7fa64eb3a640 1 --2- 172.21.15.52:0/985735175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa648144570 0x7fa648143940 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.981+0000 7fa64eb3a640 1 --2- 172.21.15.52:0/985735175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa648141010 0x7fa6480d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.981+0000 7fa64eb3a640 1 -- 172.21.15.52:0/985735175 >> 172.21.15.52:0/985735175 conn(0x7fa648072be0 msgr2=0x7fa6480d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:57.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.981+0000 7fa64eb3a640 1 -- 172.21.15.52:0/985735175 shutdown_connections 2024-05-04T12:18:57.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.981+0000 7fa64eb3a640 1 -- 172.21.15.52:0/985735175 wait complete. 2024-05-04T12:18:57.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64eb3a640 1 Processor -- start 2024-05-04T12:18:57.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64eb3a640 1 -- start start 2024-05-04T12:18:57.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64eb3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6480d2540 0x7fa648152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64eb3a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa648141010 0x7fa648160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64eb3a640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa648144570 0x7fa6481fa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64eb3a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa648141600 con 0x7fa6480d2540 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64eb3a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa648142ad0 con 0x7fa648141010 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64eb3a640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa64816b150 con 0x7fa648144570 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64c8d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6480d2540 0x7fa648152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64c8d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6480d2540 0x7fa648152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37564/0 (socket says 172.21.15.52:37564) 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64c8d6640 1 -- 172.21.15.52:0/2658663131 learned_addr learned my addr 172.21.15.52:0/2658663131 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64d0d7640 1 --2- 172.21.15.52:0/2658663131 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa648144570 0x7fa6481fa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa647fff640 1 --2- 172.21.15.52:0/2658663131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa648141010 0x7fa648160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64c8d6640 1 -- 172.21.15.52:0/2658663131 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa648144570 msgr2=0x7fa6481fa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64c8d6640 1 --2- 172.21.15.52:0/2658663131 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa648144570 0x7fa6481fa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64c8d6640 1 -- 172.21.15.52:0/2658663131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa648141010 msgr2=0x7fa648160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64c8d6640 1 --2- 172.21.15.52:0/2658663131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa648141010 0x7fa648160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64c8d6640 1 -- 172.21.15.52:0/2658663131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa648153a20 con 0x7fa6480d2540 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64c8d6640 1 --2- 172.21.15.52:0/2658663131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6480d2540 0x7fa648152450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa63800c7f0 tx=0x7fa63800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa6457fa640 1 -- 172.21.15.52:0/2658663131 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638015070 con 0x7fa6480d2540 2024-05-04T12:18:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa6457fa640 1 -- 172.21.15.52:0/2658663131 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa638004240 con 0x7fa6480d2540 2024-05-04T12:18:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa6457fa640 1 -- 172.21.15.52:0/2658663131 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638003620 con 0x7fa6480d2540 2024-05-04T12:18:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64eb3a640 1 -- 172.21.15.52:0/2658663131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa648151970 con 0x7fa6480d2540 2024-05-04T12:18:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64eb3a640 1 -- 172.21.15.52:0/2658663131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa648152990 con 0x7fa6480d2540 2024-05-04T12:18:57.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.985+0000 7fa64eb3a640 1 -- 172.21.15.52:0/2658663131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa648062e50 con 0x7fa6480d2540 2024-05-04T12:18:57.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.989+0000 7fa6457fa640 1 -- 172.21.15.52:0/2658663131 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa6380039d0 con 0x7fa6480d2540 2024-05-04T12:18:57.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.989+0000 7fa6457fa640 1 --2- 172.21.15.52:0/2658663131 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa6340678c0 0x7fa634069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:57.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.989+0000 7fa6457fa640 1 -- 172.21.15.52:0/2658663131 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa638085df0 con 0x7fa6480d2540 2024-05-04T12:18:57.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.989+0000 7fa647fff640 1 --2- 172.21.15.52:0/2658663131 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa6340678c0 0x7fa634069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:57.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.989+0000 7fa647fff640 1 --2- 172.21.15.52:0/2658663131 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa6340678c0 0x7fa634069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0ad000 tx=0x7fa63c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:57.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:57.993+0000 7fa6457fa640 1 -- 172.21.15.52:0/2658663131 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa6380568a0 con 0x7fa6480d2540 2024-05-04T12:18:58.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.157+0000 7fa64eb3a640 1 -- 172.21.15.52:0/2658663131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa648075320 con 0x7fa6480d2540 2024-05-04T12:18:58.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.161+0000 7fa6457fa640 1 -- 172.21.15.52:0/2658663131 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fa6380561a0 con 0x7fa6480d2540 2024-05-04T12:18:58.163 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:18:58.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.161+0000 7fa64eb3a640 1 -- 172.21.15.52:0/2658663131 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa6340678c0 msgr2=0x7fa634069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.161+0000 7fa64eb3a640 1 --2- 172.21.15.52:0/2658663131 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa6340678c0 0x7fa634069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0ad000 tx=0x7fa63c008040 comp rx=0 tx=0).stop 2024-05-04T12:18:58.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.161+0000 7fa64eb3a640 1 -- 172.21.15.52:0/2658663131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6480d2540 msgr2=0x7fa648152450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.161+0000 7fa64eb3a640 1 --2- 172.21.15.52:0/2658663131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6480d2540 0x7fa648152450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa63800c7f0 tx=0x7fa63800ccb0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.161+0000 7fa64eb3a640 1 -- 172.21.15.52:0/2658663131 shutdown_connections 2024-05-04T12:18:58.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.161+0000 7fa64eb3a640 1 --2- 172.21.15.52:0/2658663131 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa6340678c0 0x7fa634069d80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.161+0000 7fa64eb3a640 1 --2- 172.21.15.52:0/2658663131 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa648144570 0x7fa6481fa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.161+0000 7fa64eb3a640 1 --2- 172.21.15.52:0/2658663131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa648141010 0x7fa648160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.161+0000 7fa64eb3a640 1 --2- 172.21.15.52:0/2658663131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6480d2540 0x7fa648152450 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.161+0000 7fa64eb3a640 1 -- 172.21.15.52:0/2658663131 >> 172.21.15.52:0/2658663131 conn(0x7fa648072be0 msgr2=0x7fa64815ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:58.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.161+0000 7fa64eb3a640 1 -- 172.21.15.52:0/2658663131 shutdown_connections 2024-05-04T12:18:58.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.161+0000 7fa64eb3a640 1 -- 172.21.15.52:0/2658663131 wait complete. 2024-05-04T12:18:58.182 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:18:58.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c9b6d640 1 Processor -- start 2024-05-04T12:18:58.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c9b6d640 1 -- start start 2024-05-04T12:18:58.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c9b6d640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77c406cdd0 0x7f77c406d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:58.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c9b6d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f77c415ea70 0x7f77c40d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:58.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c9b6d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77c4072b90 con 0x7f77c4147280 2024-05-04T12:18:58.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c9b6d640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f77c4147660 con 0x7f77c415ea70 2024-05-04T12:18:58.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c9b6d640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f77c415ef00 con 0x7f77c406cdd0 2024-05-04T12:18:58.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c3fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f77c415ea70 0x7f77c40d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:58.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c3fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f77c415ea70 0x7f77c40d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37568/0 (socket says 172.21.15.52:37568) 2024-05-04T12:18:58.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c3fff640 1 -- 172.21.15.52:0/666844641 learned_addr learned my addr 172.21.15.52:0/666844641 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:58.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c2ffd640 1 --2- 172.21.15.52:0/666844641 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77c406cdd0 0x7f77c406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:58.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c3fff640 1 -- 172.21.15.52:0/666844641 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77c406cdd0 msgr2=0x7f77c406d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:58.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c3fff640 1 --2- 172.21.15.52:0/666844641 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77c406cdd0 0x7f77c406d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c3fff640 1 -- 172.21.15.52:0/666844641 >> v1:172.21.15.111:6789/0 conn(0x7f77c4147280 legacy=0x7f77c406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.305+0000 7f77c3fff640 1 -- 172.21.15.52:0/666844641 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c40725d0 con 0x7f77c415ea70 2024-05-04T12:18:58.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c3fff640 1 --2- 172.21.15.52:0/666844641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77c415ea70 0x7f77c40d0bb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009e50 tx=0x7f77bc01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2631f00c9aefd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:58.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c27fc640 1 -- 172.21.15.52:0/666844641 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0a58d0 con 0x7f77c415ea70 2024-05-04T12:18:58.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c27fc640 1 -- 172.21.15.52:0/666844641 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77bc0a5a70 con 0x7f77c415ea70 2024-05-04T12:18:58.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c27fc640 1 -- 172.21.15.52:0/666844641 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0a5e40 con 0x7f77c415ea70 2024-05-04T12:18:58.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 -- 172.21.15.52:0/666844641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77c415ea70 msgr2=0x7f77c40d0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 --2- 172.21.15.52:0/666844641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77c415ea70 0x7f77c40d0bb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009e50 tx=0x7f77bc01ca00 comp rx=0 tx=0).stop 2024-05-04T12:18:58.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 -- 172.21.15.52:0/666844641 shutdown_connections 2024-05-04T12:18:58.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 --2- 172.21.15.52:0/666844641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77c415ea70 0x7f77c40d0bb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 --2- 172.21.15.52:0/666844641 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77c406cdd0 0x7f77c406d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 -- 172.21.15.52:0/666844641 >> 172.21.15.52:0/666844641 conn(0x7f77c406b210 msgr2=0x7f77c406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:58.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 -- 172.21.15.52:0/666844641 shutdown_connections 2024-05-04T12:18:58.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 -- 172.21.15.52:0/666844641 wait complete. 2024-05-04T12:18:58.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 Processor -- start 2024-05-04T12:18:58.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 -- start start 2024-05-04T12:18:58.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77c406cdd0 0x7f77c41399f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:58.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c4147280 0x7f77c4160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:58.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77c415ea70 0x7f77c4160780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:58.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41480a0 con 0x7f77c415ea70 2024-05-04T12:18:58.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4147f20 con 0x7f77c4147280 2024-05-04T12:18:58.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.309+0000 7f77c9b6d640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f77c40721c0 con 0x7f77c406cdd0 2024-05-04T12:18:58.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77c415ea70 0x7f77c4160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:58.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77c415ea70 0x7f77c4160780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37584/0 (socket says 172.21.15.52:37584) 2024-05-04T12:18:58.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c3fff640 1 -- 172.21.15.52:0/3184195199 learned_addr learned my addr 172.21.15.52:0/3184195199 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:58.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c37fe640 1 --2- 172.21.15.52:0/3184195199 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77c406cdd0 0x7f77c41399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:58.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c2ffd640 1 --2- 172.21.15.52:0/3184195199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c4147280 0x7f77c4160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:58.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c3fff640 1 -- 172.21.15.52:0/3184195199 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77c406cdd0 msgr2=0x7f77c41399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c3fff640 1 --2- 172.21.15.52:0/3184195199 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77c406cdd0 0x7f77c41399f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c3fff640 1 -- 172.21.15.52:0/3184195199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c4147280 msgr2=0x7f77c4160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c3fff640 1 --2- 172.21.15.52:0/3184195199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c4147280 0x7f77c4160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c3fff640 1 -- 172.21.15.52:0/3184195199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c4161000 con 0x7f77c415ea70 2024-05-04T12:18:58.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c3fff640 1 --2- 172.21.15.52:0/3184195199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77c415ea70 0x7f77c4160780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009e50 tx=0x7f77bc0197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:58.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77bbfff640 1 -- 172.21.15.52:0/3184195199 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0bc070 con 0x7f77c415ea70 2024-05-04T12:18:58.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77bbfff640 1 -- 172.21.15.52:0/3184195199 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77bc0b8440 con 0x7f77c415ea70 2024-05-04T12:18:58.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c9b6d640 1 -- 172.21.15.52:0/3184195199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c413b120 con 0x7f77c415ea70 2024-05-04T12:18:58.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77bbfff640 1 -- 172.21.15.52:0/3184195199 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0a58d0 con 0x7f77c415ea70 2024-05-04T12:18:58.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c9b6d640 1 -- 172.21.15.52:0/3184195199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c413b6e0 con 0x7f77c415ea70 2024-05-04T12:18:58.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c9b6d640 1 -- 172.21.15.52:0/3184195199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c4062e50 con 0x7f77c415ea70 2024-05-04T12:18:58.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77bbfff640 1 -- 172.21.15.52:0/3184195199 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f77bc0bf070 con 0x7f77c415ea70 2024-05-04T12:18:58.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77bbfff640 1 --2- 172.21.15.52:0/3184195199 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f77a40678c0 0x7f77a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:58.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77bbfff640 1 -- 172.21.15.52:0/3184195199 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f77bc120fc0 con 0x7f77c415ea70 2024-05-04T12:18:58.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c37fe640 1 --2- 172.21.15.52:0/3184195199 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f77a40678c0 0x7f77a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:58.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.313+0000 7f77c37fe640 1 --2- 172.21.15.52:0/3184195199 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f77a40678c0 0x7f77a4069d80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f77ac001650 tx=0x7f77ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:58.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.317+0000 7f77bbfff640 1 -- 172.21.15.52:0/3184195199 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f77bc016620 con 0x7f77c415ea70 2024-05-04T12:18:58.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.477+0000 7f77c9b6d640 1 -- 172.21.15.52:0/3184195199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f77c4075320 con 0x7f77c415ea70 2024-05-04T12:18:58.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.477+0000 7f77bbfff640 1 -- 172.21.15.52:0/3184195199 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f77bc0f1b90 con 0x7f77c415ea70 2024-05-04T12:18:58.481 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:18:58.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.481+0000 7f77c9b6d640 1 -- 172.21.15.52:0/3184195199 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f77a40678c0 msgr2=0x7f77a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.481+0000 7f77c9b6d640 1 --2- 172.21.15.52:0/3184195199 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f77a40678c0 0x7f77a4069d80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f77ac001650 tx=0x7f77ac009040 comp rx=0 tx=0).stop 2024-05-04T12:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.481+0000 7f77c9b6d640 1 -- 172.21.15.52:0/3184195199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77c415ea70 msgr2=0x7f77c4160780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.481+0000 7f77c9b6d640 1 --2- 172.21.15.52:0/3184195199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77c415ea70 0x7f77c4160780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009e50 tx=0x7f77bc0197a0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.481+0000 7f77c9b6d640 1 -- 172.21.15.52:0/3184195199 shutdown_connections 2024-05-04T12:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.481+0000 7f77c9b6d640 1 --2- 172.21.15.52:0/3184195199 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f77a40678c0 0x7f77a4069d80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.481+0000 7f77c9b6d640 1 --2- 172.21.15.52:0/3184195199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77c415ea70 0x7f77c4160780 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.481+0000 7f77c9b6d640 1 --2- 172.21.15.52:0/3184195199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c4147280 0x7f77c4160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.481+0000 7f77c9b6d640 1 --2- 172.21.15.52:0/3184195199 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77c406cdd0 0x7f77c41399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.481+0000 7f77c9b6d640 1 -- 172.21.15.52:0/3184195199 >> 172.21.15.52:0/3184195199 conn(0x7f77c406b210 msgr2=0x7f77c41404d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.481+0000 7f77c9b6d640 1 -- 172.21.15.52:0/3184195199 shutdown_connections 2024-05-04T12:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.481+0000 7f77c9b6d640 1 -- 172.21.15.52:0/3184195199 wait complete. 2024-05-04T12:18:58.501 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-04T12:18:58.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.625+0000 7faa03f69640 1 Processor -- start 2024-05-04T12:18:58.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.625+0000 7faa03f69640 1 -- start start 2024-05-04T12:18:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.625+0000 7faa03f69640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9fc0b5810 0x7fa9fc165630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.625+0000 7faa03f69640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9fc0cea70 con 0x7fa9fc15a630 2024-05-04T12:18:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.625+0000 7faa03f69640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9fc0ced10 con 0x7fa9fc071070 2024-05-04T12:18:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.625+0000 7faa03f69640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa9fc0cf640 con 0x7fa9fc0b5810 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.625+0000 7faa01d05640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa9fc15a630 0x7fa9fc06e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49854/0 (socket says 172.21.15.52:49854) 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.625+0000 7faa01d05640 1 -- 172.21.15.52:0/3190862857 learned_addr learned my addr 172.21.15.52:0/3190862857 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa02506640 1 --2- 172.21.15.52:0/3190862857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9fc0b5810 0x7fa9fc165630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 -- 172.21.15.52:0/3190862857 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2709280443 0 0) 0x7fa9fc0cea70 con 0x7fa9fc15a630 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 -- 172.21.15.52:0/3190862857 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9e8003530 con 0x7fa9fc15a630 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 -- 172.21.15.52:0/3190862857 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3434496969 0 0) 0x7fa9e8003530 con 0x7fa9fc15a630 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 -- 172.21.15.52:0/3190862857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9fc0b5810 msgr2=0x7fa9fc165630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 --2- 172.21.15.52:0/3190862857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9fc0b5810 0x7fa9fc165630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 -- 172.21.15.52:0/3190862857 >> v1:172.21.15.111:6789/0 conn(0x7fa9fc071070 legacy=0x7fa9fc0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 -- 172.21.15.52:0/3190862857 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9fc0ceff0 con 0x7fa9fc15a630 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 -- 172.21.15.52:0/3190862857 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7fa9f80016a0 con 0x7fa9fc15a630 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 -- 172.21.15.52:0/3190862857 >> v1:172.21.15.52:6789/0 conn(0x7fa9fc15a630 legacy=0x7fa9fc06e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 --2- 172.21.15.52:0/3190862857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9e80039f0 0x7fa9e8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 --2- 172.21.15.52:0/3190862857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9fc0b5810 0x7fa9e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 --2- 172.21.15.52:0/3190862857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9e8004d50 0x7fa9e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 -- 172.21.15.52:0/3190862857 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc0cf640 con 0x7fa9e80039f0 2024-05-04T12:18:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 -- 172.21.15.52:0/3190862857 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e800bb30 con 0x7fa9e8004d50 2024-05-04T12:18:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa01d05640 1 --2- 172.21.15.52:0/3190862857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9e80039f0 0x7fa9e8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 -- 172.21.15.52:0/3190862857 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e800bd30 con 0x7fa9fc0b5810 2024-05-04T12:18:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa01504640 1 --2- 172.21.15.52:0/3190862857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9fc0b5810 0x7fa9e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa01d05640 1 -- 172.21.15.52:0/3190862857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9fc0b5810 msgr2=0x7fa9e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa02506640 1 --2- 172.21.15.52:0/3190862857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9e8004d50 0x7fa9e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa01d05640 1 --2- 172.21.15.52:0/3190862857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9fc0b5810 0x7fa9e8004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa01d05640 1 -- 172.21.15.52:0/3190862857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9e8004d50 msgr2=0x7fa9e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa01d05640 1 --2- 172.21.15.52:0/3190862857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9e8004d50 0x7fa9e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa01d05640 1 -- 172.21.15.52:0/3190862857 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e8003530 con 0x7fa9e80039f0 2024-05-04T12:18:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa02506640 1 --2- 172.21.15.52:0/3190862857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9e8004d50 0x7fa9e80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa01d05640 1 --2- 172.21.15.52:0/3190862857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9e80039f0 0x7fa9e8003e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8007ce0 tx=0x7fa9f8020f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8e659e6929db49 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 -- 172.21.15.52:0/3190862857 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8007d20 con 0x7fa9e80039f0 2024-05-04T12:18:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 -- 172.21.15.52:0/3190862857 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f80ad020 con 0x7fa9e80039f0 2024-05-04T12:18:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa00d03640 1 -- 172.21.15.52:0/3190862857 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80a9c30 con 0x7fa9e80039f0 2024-05-04T12:18:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa03f69640 1 -- 172.21.15.52:0/3190862857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9e80039f0 msgr2=0x7fa9e8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa03f69640 1 --2- 172.21.15.52:0/3190862857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9e80039f0 0x7fa9e8003e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8007ce0 tx=0x7fa9f8020f90 comp rx=0 tx=0).stop 2024-05-04T12:18:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa02506640 1 -- 172.21.15.52:0/3190862857 reap_dead start 2024-05-04T12:18:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa03f69640 1 -- 172.21.15.52:0/3190862857 shutdown_connections 2024-05-04T12:18:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.629+0000 7faa03f69640 1 -- 172.21.15.52:0/3190862857 >> 172.21.15.52:0/3190862857 conn(0x7fa9fc0b3190 msgr2=0x7fa9fc0b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa03f69640 1 -- 172.21.15.52:0/3190862857 shutdown_connections 2024-05-04T12:18:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa03f69640 1 -- 172.21.15.52:0/3190862857 wait complete. 2024-05-04T12:18:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa03f69640 1 Processor -- start 2024-05-04T12:18:58.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa03f69640 1 -- start start 2024-05-04T12:18:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa03f69640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9e80039f0 0x7fa9fc15ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa03f69640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9fc13a4e0 0x7fa9fc15b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa03f69640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9fc15bb70 0x7fa9fc1f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa03f69640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc1659c0 con 0x7fa9fc13a4e0 2024-05-04T12:18:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa03f69640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc165840 con 0x7fa9e80039f0 2024-05-04T12:18:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa03f69640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa9fc165b40 con 0x7fa9fc15bb70 2024-05-04T12:18:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa01504640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9fc13a4e0 0x7fa9fc15b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa01504640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9fc13a4e0 0x7fa9fc15b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37608/0 (socket says 172.21.15.52:37608) 2024-05-04T12:18:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa01504640 1 -- 172.21.15.52:0/3059202033 learned_addr learned my addr 172.21.15.52:0/3059202033 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa02506640 1 --2- 172.21.15.52:0/3059202033 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9fc15bb70 0x7fa9fc1f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa01d05640 1 --2- 172.21.15.52:0/3059202033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9e80039f0 0x7fa9fc15ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa01504640 1 -- 172.21.15.52:0/3059202033 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9fc15bb70 msgr2=0x7fa9fc1f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa01504640 1 --2- 172.21.15.52:0/3059202033 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9fc15bb70 0x7fa9fc1f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa01504640 1 -- 172.21.15.52:0/3059202033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9e80039f0 msgr2=0x7fa9fc15ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa01504640 1 --2- 172.21.15.52:0/3059202033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9e80039f0 0x7fa9fc15ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa01504640 1 -- 172.21.15.52:0/3059202033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9fc15da10 con 0x7fa9fc13a4e0 2024-05-04T12:18:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa01504640 1 --2- 172.21.15.52:0/3059202033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9fc13a4e0 0x7fa9fc15b630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00d4f0 tx=0x7fa9ec00d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7fa9f27fc640 1 -- 172.21.15.52:0/3059202033 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec005480 con 0x7fa9fc13a4e0 2024-05-04T12:18:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7fa9f27fc640 1 -- 172.21.15.52:0/3059202033 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9ec017070 con 0x7fa9fc13a4e0 2024-05-04T12:18:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa03f69640 1 -- 172.21.15.52:0/3059202033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9fc1f3530 con 0x7fa9fc13a4e0 2024-05-04T12:18:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7fa9f27fc640 1 -- 172.21.15.52:0/3059202033 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec003820 con 0x7fa9fc13a4e0 2024-05-04T12:18:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa03f69640 1 -- 172.21.15.52:0/3059202033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9fc1f3af0 con 0x7fa9fc13a4e0 2024-05-04T12:18:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.633+0000 7faa03f69640 1 -- 172.21.15.52:0/3059202033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9fc062e50 con 0x7fa9fc13a4e0 2024-05-04T12:18:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.637+0000 7fa9f27fc640 1 -- 172.21.15.52:0/3059202033 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa9ec0039c0 con 0x7fa9fc13a4e0 2024-05-04T12:18:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.637+0000 7fa9f27fc640 1 --2- 172.21.15.52:0/3059202033 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa9e00678f0 0x7fa9e0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.637+0000 7fa9f27fc640 1 -- 172.21.15.52:0/3059202033 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa9ec0859e0 con 0x7fa9fc13a4e0 2024-05-04T12:18:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.637+0000 7faa01d05640 1 --2- 172.21.15.52:0/3059202033 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa9e00678f0 0x7fa9e0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.637+0000 7faa01d05640 1 --2- 172.21.15.52:0/3059202033 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa9e00678f0 0x7fa9e0069db0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80ae000 tx=0x7fa9fc0ceee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:58.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.641+0000 7fa9f27fc640 1 -- 172.21.15.52:0/3059202033 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa9fc15da10 con 0x7fa9fc13a4e0 2024-05-04T12:18:58.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.813+0000 7faa03f69640 1 -- 172.21.15.52:0/3059202033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa9fc15bf50 con 0x7fa9fc13a4e0 2024-05-04T12:18:58.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.813+0000 7fa9f27fc640 1 -- 172.21.15.52:0/3059202033 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fa9ec00f2f0 con 0x7fa9fc13a4e0 2024-05-04T12:18:58.817 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-04T12:18:58.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.817+0000 7faa03f69640 1 -- 172.21.15.52:0/3059202033 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa9e00678f0 msgr2=0x7fa9e0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.817+0000 7faa03f69640 1 --2- 172.21.15.52:0/3059202033 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa9e00678f0 0x7fa9e0069db0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80ae000 tx=0x7fa9fc0ceee0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.817+0000 7faa03f69640 1 -- 172.21.15.52:0/3059202033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9fc13a4e0 msgr2=0x7fa9fc15b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:58.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.817+0000 7faa03f69640 1 --2- 172.21.15.52:0/3059202033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9fc13a4e0 0x7fa9fc15b630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00d4f0 tx=0x7fa9ec00d9b0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.817+0000 7faa03f69640 1 -- 172.21.15.52:0/3059202033 shutdown_connections 2024-05-04T12:18:58.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.817+0000 7faa03f69640 1 --2- 172.21.15.52:0/3059202033 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa9e00678f0 0x7fa9e0069db0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.817+0000 7faa03f69640 1 --2- 172.21.15.52:0/3059202033 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9fc15bb70 0x7fa9fc1f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.817+0000 7faa03f69640 1 --2- 172.21.15.52:0/3059202033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9fc13a4e0 0x7fa9fc15b630 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.817+0000 7faa03f69640 1 --2- 172.21.15.52:0/3059202033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9e80039f0 0x7fa9fc15ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:58.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.817+0000 7faa03f69640 1 -- 172.21.15.52:0/3059202033 >> 172.21.15.52:0/3059202033 conn(0x7fa9fc0b3190 msgr2=0x7fa9fc0b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:58.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.817+0000 7faa03f69640 1 -- 172.21.15.52:0/3059202033 shutdown_connections 2024-05-04T12:18:58.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:58.817+0000 7faa03f69640 1 -- 172.21.15.52:0/3059202033 wait complete. 2024-05-04T12:18:58.835 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi052.front.sepia.ceph.com... 2024-05-04T12:18:58.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:58.836 DEBUG:teuthology.orchestra.run.smithi052:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-04T12:18:58.858 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T12:18:58.858 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T12:18:58.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:18:58.859 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-04T12:18:58.918 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:18:58.918 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T12:18:58.925 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:18:58.925 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T12:18:58.975 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:18:59.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.097+0000 7f1535d1d640 1 Processor -- start 2024-05-04T12:18:59.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.097+0000 7f1535d1d640 1 -- start start 2024-05-04T12:18:59.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.101+0000 7f1535d1d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f15300cd190 0x7f15300723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.101+0000 7f1535d1d640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1530072aa0 con 0x7f15300ccdb0 2024-05-04T12:18:59.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.101+0000 7f1535d1d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15300708e0 con 0x7f153015c7f0 2024-05-04T12:18:59.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.101+0000 7f1535d1d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f1530070690 con 0x7f15300cd190 2024-05-04T12:18:59.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.101+0000 7f152f7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f15300ccdb0 0x7f1530158640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49862/0 (socket says 172.21.15.52:49862) 2024-05-04T12:18:59.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.101+0000 7f152f7fe640 1 -- 172.21.15.52:0/3251512730 learned_addr learned my addr 172.21.15.52:0/3251512730 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:59.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.101+0000 7f152ffff640 1 --2- 172.21.15.52:0/3251512730 >> v2:172.21.15.111:3300/0 conn(0x7f15300cd190 0x7f15300723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.101+0000 7f152e7fc640 1 -- 172.21.15.52:0/3251512730 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 866863009 0 0) 0x7f1530072aa0 con 0x7f15300ccdb0 2024-05-04T12:18:59.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.101+0000 7f152e7fc640 1 -- 172.21.15.52:0/3251512730 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1518003530 con 0x7f15300ccdb0 2024-05-04T12:18:59.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.101+0000 7f152e7fc640 1 -- 172.21.15.52:0/3251512730 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3221497082 0 0) 0x7f1518003530 con 0x7f15300ccdb0 2024-05-04T12:18:59.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.101+0000 7f152ffff640 1 -- 172.21.15.52:0/3251512730 >> v1:172.21.15.111:6789/0 conn(0x7f153015c7f0 legacy=0x7f153006de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.101+0000 7f152ffff640 1 -- 172.21.15.52:0/3251512730 >> v1:172.21.15.52:6789/0 conn(0x7f15300ccdb0 legacy=0x7f1530158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.101+0000 7f152ffff640 1 -- 172.21.15.52:0/3251512730 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1530070bc0 con 0x7f15300cd190 2024-05-04T12:18:59.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.105+0000 7f152ffff640 1 --2- 172.21.15.52:0/3251512730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15300cd190 0x7f15300723e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1528009d90 tx=0x7f152801c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e95d8a4d52d322c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:59.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.105+0000 7f152e7fc640 1 -- 172.21.15.52:0/3251512730 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15280ad070 con 0x7f15300cd190 2024-05-04T12:18:59.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.105+0000 7f152e7fc640 1 -- 172.21.15.52:0/3251512730 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15280a9470 con 0x7f15300cd190 2024-05-04T12:18:59.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.105+0000 7f152e7fc640 1 -- 172.21.15.52:0/3251512730 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520001680 con 0x7f15300cd190 2024-05-04T12:18:59.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.105+0000 7f1535d1d640 1 -- 172.21.15.52:0/3251512730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15300cd190 msgr2=0x7f15300723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.105+0000 7f1535d1d640 1 --2- 172.21.15.52:0/3251512730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15300cd190 0x7f15300723e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1528009d90 tx=0x7f152801c6b0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.105+0000 7f1535d1d640 1 -- 172.21.15.52:0/3251512730 shutdown_connections 2024-05-04T12:18:59.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.105+0000 7f1535d1d640 1 --2- 172.21.15.52:0/3251512730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15300cd190 0x7f15300723e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.105+0000 7f1535d1d640 1 -- 172.21.15.52:0/3251512730 >> 172.21.15.52:0/3251512730 conn(0x7f15300b14a0 msgr2=0x7f15300b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:59.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.105+0000 7f1535d1d640 1 -- 172.21.15.52:0/3251512730 shutdown_connections 2024-05-04T12:18:59.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.105+0000 7f1535d1d640 1 -- 172.21.15.52:0/3251512730 wait complete. 2024-05-04T12:18:59.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.105+0000 7f1535d1d640 1 Processor -- start 2024-05-04T12:18:59.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.105+0000 7f1535d1d640 1 -- start start 2024-05-04T12:18:59.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1535d1d640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f15300ccdb0 0x7f15300ab680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1535d1d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15300cd190 0x7f1530153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1535d1d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f153015c7f0 0x7f15301539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1535d1d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1530169010 con 0x7f153015c7f0 2024-05-04T12:18:59.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1535d1d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1530168e90 con 0x7f15300cd190 2024-05-04T12:18:59.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1535d1d640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f1530169190 con 0x7f15300ccdb0 2024-05-04T12:18:59.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f152ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f153015c7f0 0x7f15301539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f152ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f153015c7f0 0x7f15301539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37610/0 (socket says 172.21.15.52:37610) 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f152ffff640 1 -- 172.21.15.52:0/1516308398 learned_addr learned my addr 172.21.15.52:0/1516308398 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f152effd640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15300cd190 0x7f1530153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f152f7fe640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f15300ccdb0 0x7f15300ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f152ffff640 1 -- 172.21.15.52:0/1516308398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f15300ccdb0 msgr2=0x7f15300ab680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f152ffff640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f15300ccdb0 0x7f15300ab680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f152ffff640 1 -- 172.21.15.52:0/1516308398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15300cd190 msgr2=0x7f1530153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f152ffff640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15300cd190 0x7f1530153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f152ffff640 1 -- 172.21.15.52:0/1516308398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153013c720 con 0x7f153015c7f0 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f152effd640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15300cd190 0x7f1530153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f152ffff640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f153015c7f0 0x7f15301539d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f152801ad60 tx=0x7f1528019450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1517fff640 1 -- 172.21.15.52:0/1516308398 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15280ad040 con 0x7f153015c7f0 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1535d1d640 1 -- 172.21.15.52:0/1516308398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1530139840 con 0x7f153015c7f0 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1517fff640 1 -- 172.21.15.52:0/1516308398 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15280039a0 con 0x7f153015c7f0 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1517fff640 1 -- 172.21.15.52:0/1516308398 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528003e10 con 0x7f153015c7f0 2024-05-04T12:18:59.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1535d1d640 1 -- 172.21.15.52:0/1516308398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1530139e00 con 0x7f153015c7f0 2024-05-04T12:18:59.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1535d1d640 1 -- 172.21.15.52:0/1516308398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1530062e50 con 0x7f153015c7f0 2024-05-04T12:18:59.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1517fff640 1 -- 172.21.15.52:0/1516308398 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f15280c5060 con 0x7f153015c7f0 2024-05-04T12:18:59.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1517fff640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f151c0678c0 0x7f151c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f1517fff640 1 -- 172.21.15.52:0/1516308398 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1528122ce0 con 0x7f153015c7f0 2024-05-04T12:18:59.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f152f7fe640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f151c0678c0 0x7f151c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.109+0000 7f152f7fe640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f151c0678c0 0x7f151c069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1520001ec0 tx=0x7f1520001c30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:59.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.113+0000 7f1517fff640 1 -- 172.21.15.52:0/1516308398 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1528016480 con 0x7f153015c7f0 2024-05-04T12:18:59.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.277+0000 7f1535d1d640 1 -- 172.21.15.52:0/1516308398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1530075320 con 0x7f153015c7f0 2024-05-04T12:18:59.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.281+0000 7f1517fff640 1 -- 172.21.15.52:0/1516308398 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f15280f37a0 con 0x7f153015c7f0 2024-05-04T12:18:59.284 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:18:59.284 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:18:59.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.281+0000 7f1535d1d640 1 -- 172.21.15.52:0/1516308398 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f151c0678c0 msgr2=0x7f151c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.281+0000 7f1535d1d640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f151c0678c0 0x7f151c069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1520001ec0 tx=0x7f1520001c30 comp rx=0 tx=0).stop 2024-05-04T12:18:59.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.281+0000 7f1535d1d640 1 -- 172.21.15.52:0/1516308398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f153015c7f0 msgr2=0x7f15301539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.281+0000 7f1535d1d640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f153015c7f0 0x7f15301539d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f152801ad60 tx=0x7f1528019450 comp rx=0 tx=0).stop 2024-05-04T12:18:59.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.281+0000 7f1535d1d640 1 -- 172.21.15.52:0/1516308398 shutdown_connections 2024-05-04T12:18:59.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.281+0000 7f1535d1d640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f151c0678c0 0x7f151c069d80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.281+0000 7f1535d1d640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f153015c7f0 0x7f15301539d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.281+0000 7f1535d1d640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15300cd190 0x7f1530153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.281+0000 7f1535d1d640 1 --2- 172.21.15.52:0/1516308398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f15300ccdb0 0x7f15300ab680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.281+0000 7f1535d1d640 1 -- 172.21.15.52:0/1516308398 >> 172.21.15.52:0/1516308398 conn(0x7f15300b14a0 msgr2=0x7f15300ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:59.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.281+0000 7f1535d1d640 1 -- 172.21.15.52:0/1516308398 shutdown_connections 2024-05-04T12:18:59.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.281+0000 7f1535d1d640 1 -- 172.21.15.52:0/1516308398 wait complete. 2024-05-04T12:18:59.304 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:18:59.304 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:18:59.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.421+0000 7ffb90be1640 1 Processor -- start 2024-05-04T12:18:59.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.421+0000 7ffb90be1640 1 -- start start 2024-05-04T12:18:59.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.425+0000 7ffb90be1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ffb8c145d20 0x7ffb8c0aa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.425+0000 7ffb90be1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb8c0d0210 con 0x7ffb8c0a9480 2024-05-04T12:18:59.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.425+0000 7ffb90be1640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb8c0d04f0 con 0x7ffb8c06cdd0 2024-05-04T12:18:59.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.425+0000 7ffb90be1640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ffb8c0d1100 con 0x7ffb8c145d20 2024-05-04T12:18:59.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.425+0000 7ffb8a59c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ffb8c145d20 0x7ffb8c0aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.425+0000 7ffb81d9b640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7ffb8c0a9480 0x7ffb8c06c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49876/0 (socket says 172.21.15.52:49876) 2024-05-04T12:18:59.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.425+0000 7ffb81d9b640 1 -- 172.21.15.52:0/1666829922 learned_addr learned my addr 172.21.15.52:0/1666829922 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:59.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.425+0000 7ffb8a59c640 1 -- 172.21.15.52:0/1666829922 >> v1:172.21.15.111:6789/0 conn(0x7ffb8c06cdd0 legacy=0x7ffb8c15d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.425+0000 7ffb8a59c640 1 -- 172.21.15.52:0/1666829922 >> v1:172.21.15.52:6789/0 conn(0x7ffb8c0a9480 legacy=0x7ffb8c06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.425+0000 7ffb8a59c640 1 -- 172.21.15.52:0/1666829922 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c0d07d0 con 0x7ffb8c145d20 2024-05-04T12:18:59.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.425+0000 7ffb8a59c640 1 --2- 172.21.15.52:0/1666829922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb8c145d20 0x7ffb8c0aa370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0097c0 tx=0x7ffb7c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b9797426c63e19 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:59.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.425+0000 7ffb89d9b640 1 -- 172.21.15.52:0/1666829922 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0b0070 con 0x7ffb8c145d20 2024-05-04T12:18:59.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.425+0000 7ffb89d9b640 1 -- 172.21.15.52:0/1666829922 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c0ab600 con 0x7ffb8c145d20 2024-05-04T12:18:59.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb89d9b640 1 -- 172.21.15.52:0/1666829922 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0ab9d0 con 0x7ffb8c145d20 2024-05-04T12:18:59.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 -- 172.21.15.52:0/1666829922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb8c145d20 msgr2=0x7ffb8c0aa370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 --2- 172.21.15.52:0/1666829922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb8c145d20 0x7ffb8c0aa370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0097c0 tx=0x7ffb7c01c410 comp rx=0 tx=0).stop 2024-05-04T12:18:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 -- 172.21.15.52:0/1666829922 shutdown_connections 2024-05-04T12:18:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 --2- 172.21.15.52:0/1666829922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb8c145d20 0x7ffb8c0aa370 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 -- 172.21.15.52:0/1666829922 >> 172.21.15.52:0/1666829922 conn(0x7ffb8c06b210 msgr2=0x7ffb8c06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 -- 172.21.15.52:0/1666829922 shutdown_connections 2024-05-04T12:18:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 -- 172.21.15.52:0/1666829922 wait complete. 2024-05-04T12:18:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 Processor -- start 2024-05-04T12:18:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 -- start start 2024-05-04T12:18:59.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb8c06cdd0 0x7ffb8c143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb8c0a9480 0x7ffb8c144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb8c145d20 0x7ffb8c141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c0d1710 con 0x7ffb8c145d20 2024-05-04T12:18:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c0d1590 con 0x7ffb8c06cdd0 2024-05-04T12:18:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8c16a1a0 con 0x7ffb8c0a9480 2024-05-04T12:18:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb8ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb8c145d20 0x7ffb8c141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb8ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb8c145d20 0x7ffb8c141c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37636/0 (socket says 172.21.15.52:37636) 2024-05-04T12:18:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb8ad9d640 1 -- 172.21.15.52:0/932572509 learned_addr learned my addr 172.21.15.52:0/932572509 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb81d9b640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb8c0a9480 0x7ffb8c144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb8a59c640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb8c06cdd0 0x7ffb8c143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb8ad9d640 1 -- 172.21.15.52:0/932572509 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb8c0a9480 msgr2=0x7ffb8c144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb8ad9d640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb8c0a9480 0x7ffb8c144150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb8ad9d640 1 -- 172.21.15.52:0/932572509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb8c06cdd0 msgr2=0x7ffb8c143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb8ad9d640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb8c06cdd0 0x7ffb8c143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb8ad9d640 1 -- 172.21.15.52:0/932572509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c1429c0 con 0x7ffb8c145d20 2024-05-04T12:18:59.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb81d9b640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb8c0a9480 0x7ffb8c144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:59.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb8a59c640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb8c06cdd0 0x7ffb8c143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:59.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb8ad9d640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb8c145d20 0x7ffb8c141c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400c7a0 tx=0x7ffb8400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:59.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb837fe640 1 -- 172.21.15.52:0/932572509 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84003b80 con 0x7ffb8c145d20 2024-05-04T12:18:59.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb837fe640 1 -- 172.21.15.52:0/932572509 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb84003d20 con 0x7ffb8c145d20 2024-05-04T12:18:59.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 -- 172.21.15.52:0/932572509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8c142ca0 con 0x7ffb8c145d20 2024-05-04T12:18:59.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb837fe640 1 -- 172.21.15.52:0/932572509 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8400aa30 con 0x7ffb8c145d20 2024-05-04T12:18:59.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.429+0000 7ffb90be1640 1 -- 172.21.15.52:0/932572509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb8c143260 con 0x7ffb8c145d20 2024-05-04T12:18:59.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.433+0000 7ffb90be1640 1 -- 172.21.15.52:0/932572509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb8c062e50 con 0x7ffb8c145d20 2024-05-04T12:18:59.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.433+0000 7ffb837fe640 1 -- 172.21.15.52:0/932572509 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffb8400abd0 con 0x7ffb8c145d20 2024-05-04T12:18:59.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.433+0000 7ffb837fe640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ffb6c0678f0 0x7ffb6c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.433+0000 7ffb837fe640 1 -- 172.21.15.52:0/932572509 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ffb84085bc0 con 0x7ffb8c145d20 2024-05-04T12:18:59.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.433+0000 7ffb8a59c640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ffb6c0678f0 0x7ffb6c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.433+0000 7ffb8a59c640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ffb6c0678f0 0x7ffb6c069db0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0ad000 tx=0x7ffb7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:59.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.437+0000 7ffb837fe640 1 -- 172.21.15.52:0/932572509 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffb84056700 con 0x7ffb8c145d20 2024-05-04T12:18:59.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.597+0000 7ffb90be1640 1 -- 172.21.15.52:0/932572509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffb8c075320 con 0x7ffb8c145d20 2024-05-04T12:18:59.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.597+0000 7ffb837fe640 1 -- 172.21.15.52:0/932572509 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7ffb84056000 con 0x7ffb8c145d20 2024-05-04T12:18:59.603 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:18:59.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.601+0000 7ffb90be1640 1 -- 172.21.15.52:0/932572509 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ffb6c0678f0 msgr2=0x7ffb6c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.601+0000 7ffb90be1640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ffb6c0678f0 0x7ffb6c069db0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0ad000 tx=0x7ffb7c008040 comp rx=0 tx=0).stop 2024-05-04T12:18:59.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.601+0000 7ffb90be1640 1 -- 172.21.15.52:0/932572509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb8c145d20 msgr2=0x7ffb8c141c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.601+0000 7ffb90be1640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb8c145d20 0x7ffb8c141c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400c7a0 tx=0x7ffb8400cc60 comp rx=0 tx=0).stop 2024-05-04T12:18:59.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.601+0000 7ffb90be1640 1 -- 172.21.15.52:0/932572509 shutdown_connections 2024-05-04T12:18:59.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.601+0000 7ffb90be1640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ffb6c0678f0 0x7ffb6c069db0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.601+0000 7ffb90be1640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb8c145d20 0x7ffb8c141c50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.601+0000 7ffb90be1640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb8c0a9480 0x7ffb8c144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.601+0000 7ffb90be1640 1 --2- 172.21.15.52:0/932572509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb8c06cdd0 0x7ffb8c143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.601+0000 7ffb90be1640 1 -- 172.21.15.52:0/932572509 >> 172.21.15.52:0/932572509 conn(0x7ffb8c06b210 msgr2=0x7ffb8c144a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:59.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.601+0000 7ffb90be1640 1 -- 172.21.15.52:0/932572509 shutdown_connections 2024-05-04T12:18:59.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.601+0000 7ffb90be1640 1 -- 172.21.15.52:0/932572509 wait complete. 2024-05-04T12:18:59.623 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:18:59.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbd5504640 1 Processor -- start 2024-05-04T12:18:59.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbd5504640 1 -- start start 2024-05-04T12:18:59.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbd5504640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdbd0141210 0x7fdbd0162970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbd5504640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdbd0070690 0x7fdbd0070b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbd5504640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbd015f450 con 0x7fdbd015e7b0 2024-05-04T12:18:59.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbd5504640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd00aee10 con 0x7fdbd0141210 2024-05-04T12:18:59.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbd5504640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd01415f0 con 0x7fdbd0070690 2024-05-04T12:18:59.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbceffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdbd0141210 0x7fdbd0162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbcf7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fdbd015e7b0 0x7fdbd015ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49892/0 (socket says 172.21.15.52:49892) 2024-05-04T12:18:59.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbcf7fe640 1 -- 172.21.15.52:0/731143106 learned_addr learned my addr 172.21.15.52:0/731143106 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:18:59.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbceffd640 1 -- 172.21.15.52:0/731143106 >> v2:172.21.15.111:3300/0 conn(0x7fdbd0070690 msgr2=0x7fdbd0070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:18:59.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbce7fc640 1 --2- 172.21.15.52:0/731143106 >> v2:172.21.15.111:3300/0 conn(0x7fdbd0070690 0x7fdbd0070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbcdffb640 1 -- 172.21.15.52:0/731143106 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3414727689 0 0) 0x7fdbd015f450 con 0x7fdbd015e7b0 2024-05-04T12:18:59.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbceffd640 1 --2- 172.21.15.52:0/731143106 >> v2:172.21.15.111:3300/0 conn(0x7fdbd0070690 0x7fdbd0070b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbceffd640 1 -- 172.21.15.52:0/731143106 >> v1:172.21.15.52:6789/0 conn(0x7fdbd015e7b0 legacy=0x7fdbd015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbceffd640 1 -- 172.21.15.52:0/731143106 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd015f730 con 0x7fdbd0141210 2024-05-04T12:18:59.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.749+0000 7fdbce7fc640 1 --2- 172.21.15.52:0/731143106 >> v2:172.21.15.111:3300/0 conn(0x7fdbd0070690 0x7fdbd0070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:18:59.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbceffd640 1 --2- 172.21.15.52:0/731143106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbd0141210 0x7fdbd0162970 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0072920 tx=0x7fdbc001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ea020cdeb29e82 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:59.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbcdffb640 1 -- 172.21.15.52:0/731143106 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00b2070 con 0x7fdbd0141210 2024-05-04T12:18:59.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbcdffb640 1 -- 172.21.15.52:0/731143106 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc00a5850 con 0x7fdbd0141210 2024-05-04T12:18:59.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbcdffb640 1 -- 172.21.15.52:0/731143106 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00a5c20 con 0x7fdbd0141210 2024-05-04T12:18:59.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 -- 172.21.15.52:0/731143106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbd0141210 msgr2=0x7fdbd0162970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 --2- 172.21.15.52:0/731143106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbd0141210 0x7fdbd0162970 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0072920 tx=0x7fdbc001c8a0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 -- 172.21.15.52:0/731143106 shutdown_connections 2024-05-04T12:18:59.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 --2- 172.21.15.52:0/731143106 >> v2:172.21.15.111:3300/0 conn(0x7fdbd0070690 0x7fdbd0070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 --2- 172.21.15.52:0/731143106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbd0141210 0x7fdbd0162970 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 -- 172.21.15.52:0/731143106 >> 172.21.15.52:0/731143106 conn(0x7fdbd0067730 msgr2=0x7fdbd00b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:59.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 -- 172.21.15.52:0/731143106 shutdown_connections 2024-05-04T12:18:59.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 -- 172.21.15.52:0/731143106 wait complete. 2024-05-04T12:18:59.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 Processor -- start 2024-05-04T12:18:59.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 -- start start 2024-05-04T12:18:59.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd0070690 0x7fdbd0156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbd0141210 0x7fdbd01570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdbd015e7b0 0x7fdbd0155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd00cc790 con 0x7fdbd0141210 2024-05-04T12:18:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd00cc610 con 0x7fdbd0070690 2024-05-04T12:18:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbd5504640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd00cc910 con 0x7fdbd015e7b0 2024-05-04T12:18:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbce7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbd0141210 0x7fdbd01570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbce7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbd0141210 0x7fdbd01570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37650/0 (socket says 172.21.15.52:37650) 2024-05-04T12:18:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.753+0000 7fdbce7fc640 1 -- 172.21.15.52:0/3438836053 learned_addr learned my addr 172.21.15.52:0/3438836053 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:18:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbcf7fe640 1 --2- 172.21.15.52:0/3438836053 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdbd015e7b0 0x7fdbd0155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbceffd640 1 --2- 172.21.15.52:0/3438836053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd0070690 0x7fdbd0156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbce7fc640 1 -- 172.21.15.52:0/3438836053 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdbd015e7b0 msgr2=0x7fdbd0155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbce7fc640 1 --2- 172.21.15.52:0/3438836053 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdbd015e7b0 0x7fdbd0155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbce7fc640 1 -- 172.21.15.52:0/3438836053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd0070690 msgr2=0x7fdbd0156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbce7fc640 1 --2- 172.21.15.52:0/3438836053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd0070690 0x7fdbd0156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbce7fc640 1 -- 172.21.15.52:0/3438836053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd0134150 con 0x7fdbd0141210 2024-05-04T12:18:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbce7fc640 1 --2- 172.21.15.52:0/3438836053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbd0141210 0x7fdbd01570e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400c7f0 tx=0x7fdbc400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbb77fe640 1 -- 172.21.15.52:0/3438836053 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4015070 con 0x7fdbd0141210 2024-05-04T12:18:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbb77fe640 1 -- 172.21.15.52:0/3438836053 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc4007820 con 0x7fdbd0141210 2024-05-04T12:18:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbd5504640 1 -- 172.21.15.52:0/3438836053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd01417b0 con 0x7fdbd0141210 2024-05-04T12:18:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbb77fe640 1 -- 172.21.15.52:0/3438836053 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4007c10 con 0x7fdbd0141210 2024-05-04T12:18:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbd5504640 1 -- 172.21.15.52:0/3438836053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd006d370 con 0x7fdbd0141210 2024-05-04T12:18:59.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbd5504640 1 -- 172.21.15.52:0/3438836053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd0062e50 con 0x7fdbd0141210 2024-05-04T12:18:59.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbb77fe640 1 -- 172.21.15.52:0/3438836053 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdbc4018070 con 0x7fdbd0141210 2024-05-04T12:18:59.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbb77fe640 1 --2- 172.21.15.52:0/3438836053 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdbbc0678c0 0x7fdbbc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:18:59.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbb77fe640 1 -- 172.21.15.52:0/3438836053 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdbc4085ca0 con 0x7fdbd0141210 2024-05-04T12:18:59.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbceffd640 1 --2- 172.21.15.52:0/3438836053 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdbbc0678c0 0x7fdbbc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:18:59.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.757+0000 7fdbceffd640 1 --2- 172.21.15.52:0/3438836053 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdbbc0678c0 0x7fdbbc069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00af000 tx=0x7fdbc00a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:18:59.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.761+0000 7fdbb77fe640 1 -- 172.21.15.52:0/3438836053 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdbc4056200 con 0x7fdbd0141210 2024-05-04T12:18:59.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.921+0000 7fdbd5504640 1 -- 172.21.15.52:0/3438836053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdbd00ae8a0 con 0x7fdbd0141210 2024-05-04T12:18:59.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.921+0000 7fdbb77fe640 1 -- 172.21.15.52:0/3438836053 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fdbc405a500 con 0x7fdbd0141210 2024-05-04T12:18:59.926 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:18:59.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.925+0000 7fdbd5504640 1 -- 172.21.15.52:0/3438836053 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdbbc0678c0 msgr2=0x7fdbbc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.925+0000 7fdbd5504640 1 --2- 172.21.15.52:0/3438836053 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdbbc0678c0 0x7fdbbc069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00af000 tx=0x7fdbc00a7070 comp rx=0 tx=0).stop 2024-05-04T12:18:59.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.925+0000 7fdbd5504640 1 -- 172.21.15.52:0/3438836053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbd0141210 msgr2=0x7fdbd01570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:18:59.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.925+0000 7fdbd5504640 1 --2- 172.21.15.52:0/3438836053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbd0141210 0x7fdbd01570e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400c7f0 tx=0x7fdbc400ccb0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.925+0000 7fdbd5504640 1 -- 172.21.15.52:0/3438836053 shutdown_connections 2024-05-04T12:18:59.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.925+0000 7fdbd5504640 1 --2- 172.21.15.52:0/3438836053 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdbbc0678c0 0x7fdbbc069d80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.925+0000 7fdbd5504640 1 --2- 172.21.15.52:0/3438836053 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdbd015e7b0 0x7fdbd0155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.925+0000 7fdbd5504640 1 --2- 172.21.15.52:0/3438836053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbd0141210 0x7fdbd01570e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.925+0000 7fdbd5504640 1 --2- 172.21.15.52:0/3438836053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd0070690 0x7fdbd0156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:18:59.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.925+0000 7fdbd5504640 1 -- 172.21.15.52:0/3438836053 >> 172.21.15.52:0/3438836053 conn(0x7fdbd0067730 msgr2=0x7fdbd0160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:18:59.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.925+0000 7fdbd5504640 1 -- 172.21.15.52:0/3438836053 shutdown_connections 2024-05-04T12:18:59.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:18:59.925+0000 7fdbd5504640 1 -- 172.21.15.52:0/3438836053 wait complete. 2024-05-04T12:18:59.943 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:19:00.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.053+0000 7f2bc78a7640 1 Processor -- start 2024-05-04T12:19:00.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.053+0000 7f2bc78a7640 1 -- start start 2024-05-04T12:19:00.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc78a7640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2bc0070da0 0x7f2bc0162970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc78a7640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bc015f450 con 0x7f2bc0141210 2024-05-04T12:19:00.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc78a7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bc015f730 con 0x7f2bc015e7b0 2024-05-04T12:19:00.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc78a7640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f2bc00aee10 con 0x7f2bc0070da0 2024-05-04T12:19:00.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc4e42640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2bc0070da0 0x7f2bc0162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc5643640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f2bc0141210 0x7f2bc0070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49904/0 (socket says 172.21.15.52:49904) 2024-05-04T12:19:00.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc5643640 1 -- 172.21.15.52:0/2988944964 learned_addr learned my addr 172.21.15.52:0/2988944964 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc4e42640 1 -- 172.21.15.52:0/2988944964 >> v1:172.21.15.111:6789/0 conn(0x7f2bc015e7b0 legacy=0x7f2bc015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc4e42640 1 -- 172.21.15.52:0/2988944964 >> v1:172.21.15.52:6789/0 conn(0x7f2bc0141210 legacy=0x7f2bc0070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc4e42640 1 -- 172.21.15.52:0/2988944964 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc0161e90 con 0x7f2bc0070da0 2024-05-04T12:19:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bb7fff640 1 -- 172.21.15.52:0/2988944964 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2549321135 0 0) 0x7f2bc015f450 con 0x7f2bc0141210 2024-05-04T12:19:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc4e42640 1 --2- 172.21.15.52:0/2988944964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bc0070da0 0x7f2bc0162970 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00097c0 tx=0x7f2bb001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56de1c94c4d6652 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bb7fff640 1 -- 172.21.15.52:0/2988944964 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00b0070 con 0x7f2bc0070da0 2024-05-04T12:19:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bb7fff640 1 -- 172.21.15.52:0/2988944964 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb00ab600 con 0x7f2bc0070da0 2024-05-04T12:19:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bb7fff640 1 -- 172.21.15.52:0/2988944964 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00ab9d0 con 0x7f2bc0070da0 2024-05-04T12:19:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc78a7640 1 -- 172.21.15.52:0/2988944964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bc0070da0 msgr2=0x7f2bc0162970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc78a7640 1 --2- 172.21.15.52:0/2988944964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bc0070da0 0x7f2bc0162970 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00097c0 tx=0x7f2bb001c410 comp rx=0 tx=0).stop 2024-05-04T12:19:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc78a7640 1 -- 172.21.15.52:0/2988944964 shutdown_connections 2024-05-04T12:19:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc78a7640 1 --2- 172.21.15.52:0/2988944964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bc0070da0 0x7f2bc0162970 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc78a7640 1 -- 172.21.15.52:0/2988944964 >> 172.21.15.52:0/2988944964 conn(0x7f2bc0067730 msgr2=0x7f2bc00b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc78a7640 1 -- 172.21.15.52:0/2988944964 shutdown_connections 2024-05-04T12:19:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.057+0000 7f2bc78a7640 1 -- 172.21.15.52:0/2988944964 wait complete. 2024-05-04T12:19:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc78a7640 1 Processor -- start 2024-05-04T12:19:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc78a7640 1 -- start start 2024-05-04T12:19:00.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc78a7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2bc0070da0 0x7f2bc01542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc78a7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bc0141210 0x7f2bc01363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc78a7640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2bc015e7b0 0x7f2bc013f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc78a7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc00cc9e0 con 0x7f2bc0141210 2024-05-04T12:19:00.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc78a7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc00cc860 con 0x7f2bc0070da0 2024-05-04T12:19:00.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc78a7640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f2bc00ccb60 con 0x7f2bc015e7b0 2024-05-04T12:19:00.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc4e42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bc0141210 0x7f2bc01363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc4e42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bc0141210 0x7f2bc01363d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37666/0 (socket says 172.21.15.52:37666) 2024-05-04T12:19:00.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc4e42640 1 -- 172.21.15.52:0/75658753 learned_addr learned my addr 172.21.15.52:0/75658753 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:00.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc5643640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2bc0070da0 0x7f2bc01542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc5e44640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2bc015e7b0 0x7f2bc013f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc4e42640 1 -- 172.21.15.52:0/75658753 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2bc015e7b0 msgr2=0x7f2bc013f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc4e42640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2bc015e7b0 0x7f2bc013f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc4e42640 1 -- 172.21.15.52:0/75658753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2bc0070da0 msgr2=0x7f2bc01542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc4e42640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2bc0070da0 0x7f2bc01542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc4e42640 1 -- 172.21.15.52:0/75658753 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc0136ab0 con 0x7f2bc0141210 2024-05-04T12:19:00.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc5e44640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2bc015e7b0 0x7f2bc013f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:00.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc5643640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2bc0070da0 0x7f2bc01542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:00.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc4e42640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bc0141210 0x7f2bc01363d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0019020 tx=0x7f2bb00191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:00.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bb5ffb640 1 -- 172.21.15.52:0/75658753 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00b8020 con 0x7f2bc0141210 2024-05-04T12:19:00.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bb5ffb640 1 -- 172.21.15.52:0/75658753 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb00b4630 con 0x7f2bc0141210 2024-05-04T12:19:00.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc78a7640 1 -- 172.21.15.52:0/75658753 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc0136d90 con 0x7f2bc0141210 2024-05-04T12:19:00.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bb5ffb640 1 -- 172.21.15.52:0/75658753 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00b0040 con 0x7f2bc0141210 2024-05-04T12:19:00.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc78a7640 1 -- 172.21.15.52:0/75658753 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc0137350 con 0x7f2bc0141210 2024-05-04T12:19:00.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bc78a7640 1 -- 172.21.15.52:0/75658753 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc0062e50 con 0x7f2bc0141210 2024-05-04T12:19:00.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.061+0000 7f2bb5ffb640 1 -- 172.21.15.52:0/75658753 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2bb00bb020 con 0x7f2bc0141210 2024-05-04T12:19:00.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.065+0000 7f2bb5ffb640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2ba40678c0 0x7f2ba4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.065+0000 7f2bb5ffb640 1 -- 172.21.15.52:0/75658753 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2bb011f9c0 con 0x7f2bc0141210 2024-05-04T12:19:00.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.065+0000 7f2bc5643640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2ba40678c0 0x7f2ba4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.065+0000 7f2bc5643640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2ba40678c0 0x7f2ba4069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc000f70 tx=0x7f2bbc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:00.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.069+0000 7f2bb5ffb640 1 -- 172.21.15.52:0/75658753 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2bb0121e20 con 0x7f2bc0141210 2024-05-04T12:19:00.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.229+0000 7f2bc78a7640 1 -- 172.21.15.52:0/75658753 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2bc0075320 con 0x7f2bc0141210 2024-05-04T12:19:00.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.229+0000 7f2bb5ffb640 1 -- 172.21.15.52:0/75658753 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f2bb00c2030 con 0x7f2bc0141210 2024-05-04T12:19:00.232 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:19:00.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.229+0000 7f2bc78a7640 1 -- 172.21.15.52:0/75658753 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2ba40678c0 msgr2=0x7f2ba4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.229+0000 7f2bc78a7640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2ba40678c0 0x7f2ba4069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc000f70 tx=0x7f2bbc009040 comp rx=0 tx=0).stop 2024-05-04T12:19:00.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.229+0000 7f2bc78a7640 1 -- 172.21.15.52:0/75658753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bc0141210 msgr2=0x7f2bc01363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.229+0000 7f2bc78a7640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bc0141210 0x7f2bc01363d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0019020 tx=0x7f2bb00191b0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.229+0000 7f2bc78a7640 1 -- 172.21.15.52:0/75658753 shutdown_connections 2024-05-04T12:19:00.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.229+0000 7f2bc78a7640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2ba40678c0 0x7f2ba4069d80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.229+0000 7f2bc78a7640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2bc015e7b0 0x7f2bc013f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.229+0000 7f2bc78a7640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bc0141210 0x7f2bc01363d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.229+0000 7f2bc78a7640 1 --2- 172.21.15.52:0/75658753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2bc0070da0 0x7f2bc01542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.229+0000 7f2bc78a7640 1 -- 172.21.15.52:0/75658753 >> 172.21.15.52:0/75658753 conn(0x7f2bc0067730 msgr2=0x7f2bc00b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:00.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.229+0000 7f2bc78a7640 1 -- 172.21.15.52:0/75658753 shutdown_connections 2024-05-04T12:19:00.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.229+0000 7f2bc78a7640 1 -- 172.21.15.52:0/75658753 wait complete. 2024-05-04T12:19:00.252 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-04T12:19:00.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f894ec1f640 1 Processor -- start 2024-05-04T12:19:00.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f894ec1f640 1 -- start start 2024-05-04T12:19:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f894ec1f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f894815bf50 0x7f894815c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f894ec1f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f89480aa0c0 0x7f89480ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f894ec1f640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f894806d1b0 con 0x7f89480aa550 2024-05-04T12:19:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f894ec1f640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f894806d490 con 0x7f894815bf50 2024-05-04T12:19:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f894ec1f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f894806d770 con 0x7f89480aa0c0 2024-05-04T12:19:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f894c9bb640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f894815bf50 0x7f894815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f894d1bc640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f89480aa550 0x7f894806e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49910/0 (socket says 172.21.15.52:49910) 2024-05-04T12:19:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f894d1bc640 1 -- 172.21.15.52:0/2538914423 learned_addr learned my addr 172.21.15.52:0/2538914423 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f8947fff640 1 --2- 172.21.15.52:0/2538914423 >> v2:172.21.15.111:3300/0 conn(0x7f89480aa0c0 0x7f89480ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f894c9bb640 1 -- 172.21.15.52:0/2538914423 >> v2:172.21.15.111:3300/0 conn(0x7f89480aa0c0 msgr2=0x7f89480ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f894c9bb640 1 --2- 172.21.15.52:0/2538914423 >> v2:172.21.15.111:3300/0 conn(0x7f89480aa0c0 0x7f89480ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f894c9bb640 1 -- 172.21.15.52:0/2538914423 >> v1:172.21.15.52:6789/0 conn(0x7f89480aa550 legacy=0x7f894806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.369+0000 7f894c9bb640 1 -- 172.21.15.52:0/2538914423 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894806da50 con 0x7f894815bf50 2024-05-04T12:19:00.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.373+0000 7f894c9bb640 1 --2- 172.21.15.52:0/2538914423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f894815bf50 0x7f894815c330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f89480b3ae0 tx=0x7f893801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6585e4a85cb7c0f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:00.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.373+0000 7f89477fe640 1 -- 172.21.15.52:0/2538914423 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89380a7070 con 0x7f894815bf50 2024-05-04T12:19:00.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.373+0000 7f89477fe640 1 -- 172.21.15.52:0/2538914423 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89380b0070 con 0x7f894815bf50 2024-05-04T12:19:00.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.373+0000 7f89477fe640 1 -- 172.21.15.52:0/2538914423 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89380ab890 con 0x7f894815bf50 2024-05-04T12:19:00.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.373+0000 7f894ec1f640 1 -- 172.21.15.52:0/2538914423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f894815bf50 msgr2=0x7f894815c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.373+0000 7f894ec1f640 1 --2- 172.21.15.52:0/2538914423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f894815bf50 0x7f894815c330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f89480b3ae0 tx=0x7f893801c450 comp rx=0 tx=0).stop 2024-05-04T12:19:00.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.373+0000 7f894ec1f640 1 -- 172.21.15.52:0/2538914423 shutdown_connections 2024-05-04T12:19:00.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.373+0000 7f894ec1f640 1 --2- 172.21.15.52:0/2538914423 >> v2:172.21.15.111:3300/0 conn(0x7f89480aa0c0 0x7f89480ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.373+0000 7f894ec1f640 1 --2- 172.21.15.52:0/2538914423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f894815bf50 0x7f894815c330 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.373+0000 7f894ec1f640 1 -- 172.21.15.52:0/2538914423 >> 172.21.15.52:0/2538914423 conn(0x7f89480b14a0 msgr2=0x7f89480b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:00.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.373+0000 7f894ec1f640 1 -- 172.21.15.52:0/2538914423 shutdown_connections 2024-05-04T12:19:00.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.373+0000 7f894ec1f640 1 -- 172.21.15.52:0/2538914423 wait complete. 2024-05-04T12:19:00.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.373+0000 7f894ec1f640 1 Processor -- start 2024-05-04T12:19:00.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.373+0000 7f894ec1f640 1 -- start start 2024-05-04T12:19:00.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f894ec1f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f89480aa0c0 0x7f8948147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f894ec1f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89480aa550 0x7f8948159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f894ec1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f894815bf50 0x7f89480cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f894ec1f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8948169030 con 0x7f89480aa550 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f894ec1f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f89480cd4a0 con 0x7f894815bf50 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f894ec1f640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f89481691b0 con 0x7f89480aa0c0 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f8947fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89480aa550 0x7f8948159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f8947fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89480aa550 0x7f8948159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37686/0 (socket says 172.21.15.52:37686) 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f8947fff640 1 -- 172.21.15.52:0/2850343511 learned_addr learned my addr 172.21.15.52:0/2850343511 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f894d1bc640 1 --2- 172.21.15.52:0/2850343511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f894815bf50 0x7f89480cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f894c9bb640 1 --2- 172.21.15.52:0/2850343511 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f89480aa0c0 0x7f8948147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f8947fff640 1 -- 172.21.15.52:0/2850343511 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f89480aa0c0 msgr2=0x7f8948147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f8947fff640 1 --2- 172.21.15.52:0/2850343511 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f89480aa0c0 0x7f8948147a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f8947fff640 1 -- 172.21.15.52:0/2850343511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f894815bf50 msgr2=0x7f89480cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f8947fff640 1 --2- 172.21.15.52:0/2850343511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f894815bf50 0x7f89480cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f8947fff640 1 -- 172.21.15.52:0/2850343511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894815a8e0 con 0x7f89480aa550 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f8947fff640 1 --2- 172.21.15.52:0/2850343511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89480aa550 0x7f8948159e90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f893c00d800 tx=0x7f893c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:00.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f89457fa640 1 -- 172.21.15.52:0/2850343511 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c00b6c0 con 0x7f89480aa550 2024-05-04T12:19:00.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f89457fa640 1 -- 172.21.15.52:0/2850343511 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f893c019070 con 0x7f89480aa550 2024-05-04T12:19:00.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f894ec1f640 1 -- 172.21.15.52:0/2850343511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894815abc0 con 0x7f89480aa550 2024-05-04T12:19:00.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f89457fa640 1 -- 172.21.15.52:0/2850343511 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c00ba60 con 0x7f89480aa550 2024-05-04T12:19:00.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f894ec1f640 1 -- 172.21.15.52:0/2850343511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894815b180 con 0x7f89480aa550 2024-05-04T12:19:00.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f894ec1f640 1 -- 172.21.15.52:0/2850343511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8948062e50 con 0x7f89480aa550 2024-05-04T12:19:00.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f89457fa640 1 -- 172.21.15.52:0/2850343511 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f893c00bc00 con 0x7f89480aa550 2024-05-04T12:19:00.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f89457fa640 1 --2- 172.21.15.52:0/2850343511 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f89340678f0 0x7f8934069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f89457fa640 1 -- 172.21.15.52:0/2850343511 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f893c086ba0 con 0x7f89480aa550 2024-05-04T12:19:00.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f894c9bb640 1 --2- 172.21.15.52:0/2850343511 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f89340678f0 0x7f8934069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.377+0000 7f894c9bb640 1 --2- 172.21.15.52:0/2850343511 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f89340678f0 0x7f8934069db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f89380af000 tx=0x7f89380a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:00.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.381+0000 7f89457fa640 1 -- 172.21.15.52:0/2850343511 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f893c0577c0 con 0x7f89480aa550 2024-05-04T12:19:00.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.545+0000 7f894ec1f640 1 -- 172.21.15.52:0/2850343511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8948075320 con 0x7f89480aa550 2024-05-04T12:19:00.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.549+0000 7f89457fa640 1 -- 172.21.15.52:0/2850343511 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f893c0570c0 con 0x7f89480aa550 2024-05-04T12:19:00.551 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-04T12:19:00.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.549+0000 7f894ec1f640 1 -- 172.21.15.52:0/2850343511 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f89340678f0 msgr2=0x7f8934069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.549+0000 7f894ec1f640 1 --2- 172.21.15.52:0/2850343511 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f89340678f0 0x7f8934069db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f89380af000 tx=0x7f89380a7000 comp rx=0 tx=0).stop 2024-05-04T12:19:00.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.549+0000 7f894ec1f640 1 -- 172.21.15.52:0/2850343511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89480aa550 msgr2=0x7f8948159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.549+0000 7f894ec1f640 1 --2- 172.21.15.52:0/2850343511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89480aa550 0x7f8948159e90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f893c00d800 tx=0x7f893c00dcc0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.549+0000 7f894ec1f640 1 -- 172.21.15.52:0/2850343511 shutdown_connections 2024-05-04T12:19:00.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.549+0000 7f894ec1f640 1 --2- 172.21.15.52:0/2850343511 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f89340678f0 0x7f8934069db0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.549+0000 7f894ec1f640 1 --2- 172.21.15.52:0/2850343511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f894815bf50 0x7f89480cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.549+0000 7f894ec1f640 1 --2- 172.21.15.52:0/2850343511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89480aa550 0x7f8948159e90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.549+0000 7f894ec1f640 1 --2- 172.21.15.52:0/2850343511 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f89480aa0c0 0x7f8948147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.549+0000 7f894ec1f640 1 -- 172.21.15.52:0/2850343511 >> 172.21.15.52:0/2850343511 conn(0x7f89480b14a0 msgr2=0x7f8948070720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:00.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.549+0000 7f894ec1f640 1 -- 172.21.15.52:0/2850343511 shutdown_connections 2024-05-04T12:19:00.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.549+0000 7f894ec1f640 1 -- 172.21.15.52:0/2850343511 wait complete. 2024-05-04T12:19:00.569 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:19:00.569 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T12:19:00.575 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:19:00.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.693+0000 7fed4a022640 1 Processor -- start 2024-05-04T12:19:00.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.693+0000 7fed4a022640 1 -- start start 2024-05-04T12:19:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed4a022640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fed4415bf50 0x7fed4415c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed4a022640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed440aa0c0 0x7fed440aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed4a022640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed4406d1b0 con 0x7fed440ccb50 2024-05-04T12:19:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed4a022640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fed4406d490 con 0x7fed4415bf50 2024-05-04T12:19:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed4a022640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fed4406d770 con 0x7fed440aa0c0 2024-05-04T12:19:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed43fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fed440ccb50 0x7fed4406e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49924/0 (socket says 172.21.15.52:49924) 2024-05-04T12:19:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed43fff640 1 -- 172.21.15.52:0/820545841 learned_addr learned my addr 172.21.15.52:0/820545841 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed42ffd640 1 --2- 172.21.15.52:0/820545841 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed440aa0c0 0x7fed440aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed437fe640 1 --2- 172.21.15.52:0/820545841 >> v2:172.21.15.111:3300/0 conn(0x7fed4415bf50 0x7fed4415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed427fc640 1 -- 172.21.15.52:0/820545841 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 421520143 0 0) 0x7fed4406d1b0 con 0x7fed440ccb50 2024-05-04T12:19:00.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed427fc640 1 -- 172.21.15.52:0/820545841 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed2c003530 con 0x7fed440ccb50 2024-05-04T12:19:00.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed437fe640 1 -- 172.21.15.52:0/820545841 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed440aa0c0 msgr2=0x7fed440aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:00.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed437fe640 1 --2- 172.21.15.52:0/820545841 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed440aa0c0 0x7fed440aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed427fc640 1 -- 172.21.15.52:0/820545841 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1458346829 0 0) 0x7fed2c003530 con 0x7fed440ccb50 2024-05-04T12:19:00.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed437fe640 1 -- 172.21.15.52:0/820545841 >> v1:172.21.15.52:6789/0 conn(0x7fed440ccb50 legacy=0x7fed4406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed437fe640 1 -- 172.21.15.52:0/820545841 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4406da50 con 0x7fed4415bf50 2024-05-04T12:19:00.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.697+0000 7fed42ffd640 1 --2- 172.21.15.52:0/820545841 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed440aa0c0 0x7fed440aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:19:00.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.701+0000 7fed437fe640 1 --2- 172.21.15.52:0/820545841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed4415bf50 0x7fed4415c330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fed340097c0 tx=0x7fed3401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61371297c8709241 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:00.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.701+0000 7fed427fc640 1 -- 172.21.15.52:0/820545841 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340b0070 con 0x7fed4415bf50 2024-05-04T12:19:00.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.701+0000 7fed427fc640 1 -- 172.21.15.52:0/820545841 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed340ab400 con 0x7fed4415bf50 2024-05-04T12:19:00.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.701+0000 7fed427fc640 1 -- 172.21.15.52:0/820545841 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c001a00 con 0x7fed4415bf50 2024-05-04T12:19:00.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.701+0000 7fed4a022640 1 -- 172.21.15.52:0/820545841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed4415bf50 msgr2=0x7fed4415c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.701+0000 7fed4a022640 1 --2- 172.21.15.52:0/820545841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed4415bf50 0x7fed4415c330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fed340097c0 tx=0x7fed3401c390 comp rx=0 tx=0).stop 2024-05-04T12:19:00.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.701+0000 7fed4a022640 1 -- 172.21.15.52:0/820545841 shutdown_connections 2024-05-04T12:19:00.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.701+0000 7fed4a022640 1 --2- 172.21.15.52:0/820545841 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed440aa0c0 0x7fed440aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.701+0000 7fed4a022640 1 --2- 172.21.15.52:0/820545841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed4415bf50 0x7fed4415c330 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.701+0000 7fed4a022640 1 -- 172.21.15.52:0/820545841 >> 172.21.15.52:0/820545841 conn(0x7fed440b14a0 msgr2=0x7fed440b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:00.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.701+0000 7fed4a022640 1 -- 172.21.15.52:0/820545841 shutdown_connections 2024-05-04T12:19:00.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.701+0000 7fed4a022640 1 -- 172.21.15.52:0/820545841 wait complete. 2024-05-04T12:19:00.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed4a022640 1 Processor -- start 2024-05-04T12:19:00.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed4a022640 1 -- start start 2024-05-04T12:19:00.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed4a022640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed440aa0c0 0x7fed4413c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed4a022640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed440ccb50 0x7fed440cf870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed4a022640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed4415bf50 0x7fed441569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed4a022640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fed4406cee0 con 0x7fed440ccb50 2024-05-04T12:19:00.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed4a022640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fed4406cd60 con 0x7fed4415bf50 2024-05-04T12:19:00.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed4a022640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fed44169030 con 0x7fed440aa0c0 2024-05-04T12:19:00.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed42ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed440ccb50 0x7fed440cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed42ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed440ccb50 0x7fed440cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37700/0 (socket says 172.21.15.52:37700) 2024-05-04T12:19:00.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed42ffd640 1 -- 172.21.15.52:0/3332771535 learned_addr learned my addr 172.21.15.52:0/3332771535 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:00.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed437fe640 1 --2- 172.21.15.52:0/3332771535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed440aa0c0 0x7fed4413c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed43fff640 1 --2- 172.21.15.52:0/3332771535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed4415bf50 0x7fed441569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed42ffd640 1 -- 172.21.15.52:0/3332771535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed440aa0c0 msgr2=0x7fed4413c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed42ffd640 1 --2- 172.21.15.52:0/3332771535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed440aa0c0 0x7fed4413c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed42ffd640 1 -- 172.21.15.52:0/3332771535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed4415bf50 msgr2=0x7fed441569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed42ffd640 1 --2- 172.21.15.52:0/3332771535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed4415bf50 0x7fed441569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed42ffd640 1 -- 172.21.15.52:0/3332771535 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4414aa90 con 0x7fed440ccb50 2024-05-04T12:19:00.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed42ffd640 1 --2- 172.21.15.52:0/3332771535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed440ccb50 0x7fed440cf870 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fed3800ebe0 tx=0x7fed3800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:00.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed2bfff640 1 -- 172.21.15.52:0/3332771535 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38017070 con 0x7fed440ccb50 2024-05-04T12:19:00.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed2bfff640 1 -- 172.21.15.52:0/3332771535 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed3800cc30 con 0x7fed440ccb50 2024-05-04T12:19:00.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed4a022640 1 -- 172.21.15.52:0/3332771535 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4414ad70 con 0x7fed440ccb50 2024-05-04T12:19:00.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed2bfff640 1 -- 172.21.15.52:0/3332771535 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38013470 con 0x7fed440ccb50 2024-05-04T12:19:00.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed4a022640 1 -- 172.21.15.52:0/3332771535 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed4414d440 con 0x7fed440ccb50 2024-05-04T12:19:00.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed4a022640 1 -- 172.21.15.52:0/3332771535 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed44062e50 con 0x7fed440ccb50 2024-05-04T12:19:00.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed2bfff640 1 -- 172.21.15.52:0/3332771535 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fed3801a070 con 0x7fed440ccb50 2024-05-04T12:19:00.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed2bfff640 1 --2- 172.21.15.52:0/3332771535 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fed300678c0 0x7fed30069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:00.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.705+0000 7fed2bfff640 1 -- 172.21.15.52:0/3332771535 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fed38085f30 con 0x7fed440ccb50 2024-05-04T12:19:00.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.709+0000 7fed437fe640 1 --2- 172.21.15.52:0/3332771535 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fed300678c0 0x7fed30069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:00.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.709+0000 7fed437fe640 1 --2- 172.21.15.52:0/3332771535 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fed300678c0 0x7fed30069d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fed340ad000 tx=0x7fed340a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:00.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.713+0000 7fed2bfff640 1 -- 172.21.15.52:0/3332771535 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fed38056b00 con 0x7fed440ccb50 2024-05-04T12:19:00.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.873+0000 7fed4a022640 1 -- 172.21.15.52:0/3332771535 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed4414d9f0 con 0x7fed440ccb50 2024-05-04T12:19:00.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.877+0000 7fed2bfff640 1 -- 172.21.15.52:0/3332771535 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fed38056400 con 0x7fed440ccb50 2024-05-04T12:19:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:00.881 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:19:00.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.877+0000 7fed4a022640 1 -- 172.21.15.52:0/3332771535 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fed300678c0 msgr2=0x7fed30069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.877+0000 7fed4a022640 1 --2- 172.21.15.52:0/3332771535 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fed300678c0 0x7fed30069d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fed340ad000 tx=0x7fed340a6000 comp rx=0 tx=0).stop 2024-05-04T12:19:00.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.877+0000 7fed4a022640 1 -- 172.21.15.52:0/3332771535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed440ccb50 msgr2=0x7fed440cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:00.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.877+0000 7fed4a022640 1 --2- 172.21.15.52:0/3332771535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed440ccb50 0x7fed440cf870 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fed3800ebe0 tx=0x7fed3800c520 comp rx=0 tx=0).stop 2024-05-04T12:19:00.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.877+0000 7fed4a022640 1 -- 172.21.15.52:0/3332771535 shutdown_connections 2024-05-04T12:19:00.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.877+0000 7fed4a022640 1 --2- 172.21.15.52:0/3332771535 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fed300678c0 0x7fed30069d80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.877+0000 7fed4a022640 1 --2- 172.21.15.52:0/3332771535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed4415bf50 0x7fed441569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.877+0000 7fed4a022640 1 --2- 172.21.15.52:0/3332771535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed440ccb50 0x7fed440cf870 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.877+0000 7fed4a022640 1 --2- 172.21.15.52:0/3332771535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed440aa0c0 0x7fed4413c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:00.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.877+0000 7fed4a022640 1 -- 172.21.15.52:0/3332771535 >> 172.21.15.52:0/3332771535 conn(0x7fed440b14a0 msgr2=0x7fed44070730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:00.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.881+0000 7fed4a022640 1 -- 172.21.15.52:0/3332771535 shutdown_connections 2024-05-04T12:19:00.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:00.881+0000 7fed4a022640 1 -- 172.21.15.52:0/3332771535 wait complete. 2024-05-04T12:19:00.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:19:00.900 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:19:01.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7fec05dc4640 1 Processor -- start 2024-05-04T12:19:01.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7fec05dc4640 1 -- start start 2024-05-04T12:19:01.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7fec05dc4640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec0015c590 0x7fec000cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7fec05dc4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fec000cccd0 0x7fec0006e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7fec05dc4640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec0006dbf0 con 0x7fec00145960 2024-05-04T12:19:01.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7fec05dc4640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fec0006cfe0 con 0x7fec000cccd0 2024-05-04T12:19:01.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7fec05dc4640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fec0006d2c0 con 0x7fec0015c590 2024-05-04T12:19:01.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7febff7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fec00145960 0x7fec00160590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49940/0 (socket says 172.21.15.52:49940) 2024-05-04T12:19:01.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7febff7fe640 1 -- 172.21.15.52:0/2328114390 learned_addr learned my addr 172.21.15.52:0/2328114390 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:01.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7febfffff640 1 --2- 172.21.15.52:0/2328114390 >> v2:172.21.15.52:3300/0 conn(0x7fec000cccd0 0x7fec0006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7febf6ffd640 1 --2- 172.21.15.52:0/2328114390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec0015c590 0x7fec000cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7febfeffd640 1 -- 172.21.15.52:0/2328114390 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3113632239 0 0) 0x7fec0006dbf0 con 0x7fec00145960 2024-05-04T12:19:01.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7febfffff640 1 -- 172.21.15.52:0/2328114390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec0015c590 msgr2=0x7fec000cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:01.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7febfffff640 1 --2- 172.21.15.52:0/2328114390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec0015c590 0x7fec000cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7febfffff640 1 -- 172.21.15.52:0/2328114390 >> v1:172.21.15.52:6789/0 conn(0x7fec00145960 legacy=0x7fec00160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.025+0000 7febfffff640 1 -- 172.21.15.52:0/2328114390 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0006d5a0 con 0x7fec000cccd0 2024-05-04T12:19:01.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7febfffff640 1 --2- 172.21.15.52:0/2328114390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec000cccd0 0x7fec0006e080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7febf8009d90 tx=0x7febf801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f65ca6a4a01b634 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:01.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7febfeffd640 1 -- 172.21.15.52:0/2328114390 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf80b2070 con 0x7fec000cccd0 2024-05-04T12:19:01.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7febfeffd640 1 -- 172.21.15.52:0/2328114390 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf80a58b0 con 0x7fec000cccd0 2024-05-04T12:19:01.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7febfeffd640 1 -- 172.21.15.52:0/2328114390 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf80a5c80 con 0x7fec000cccd0 2024-05-04T12:19:01.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 -- 172.21.15.52:0/2328114390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec000cccd0 msgr2=0x7fec0006e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 --2- 172.21.15.52:0/2328114390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec000cccd0 0x7fec0006e080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7febf8009d90 tx=0x7febf801c8e0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 -- 172.21.15.52:0/2328114390 shutdown_connections 2024-05-04T12:19:01.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 --2- 172.21.15.52:0/2328114390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec000cccd0 0x7fec0006e080 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 --2- 172.21.15.52:0/2328114390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec0015c590 0x7fec000cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 -- 172.21.15.52:0/2328114390 >> 172.21.15.52:0/2328114390 conn(0x7fec00067730 msgr2=0x7fec000b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:01.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 -- 172.21.15.52:0/2328114390 shutdown_connections 2024-05-04T12:19:01.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 -- 172.21.15.52:0/2328114390 wait complete. 2024-05-04T12:19:01.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 Processor -- start 2024-05-04T12:19:01.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 -- start start 2024-05-04T12:19:01.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec000cccd0 0x7fec00071780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec00145960 0x7fec00144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec0015c590 0x7fec0015b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fec00145ec0 con 0x7fec000cccd0 2024-05-04T12:19:01.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fec00145d40 con 0x7fec0015c590 2024-05-04T12:19:01.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7fec05dc4640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fec0006edd0 con 0x7fec00145960 2024-05-04T12:19:01.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7febff7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec000cccd0 0x7fec00071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7febff7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec000cccd0 0x7fec00071780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37722/0 (socket says 172.21.15.52:37722) 2024-05-04T12:19:01.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.029+0000 7febff7fe640 1 -- 172.21.15.52:0/2543412487 learned_addr learned my addr 172.21.15.52:0/2543412487 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:01.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febf6ffd640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec00145960 0x7fec00144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febfffff640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec0015c590 0x7fec0015b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febff7fe640 1 -- 172.21.15.52:0/2543412487 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec00145960 msgr2=0x7fec00144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febff7fe640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec00145960 0x7fec00144bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febff7fe640 1 -- 172.21.15.52:0/2543412487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec0015c590 msgr2=0x7fec0015b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febff7fe640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec0015c590 0x7fec0015b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febff7fe640 1 -- 172.21.15.52:0/2543412487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec00071e80 con 0x7fec000cccd0 2024-05-04T12:19:01.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febfffff640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec0015c590 0x7fec0015b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:01.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febff7fe640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec000cccd0 0x7fec00071780 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7febf00011f0 tx=0x7febf000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:01.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febfcff9640 1 -- 172.21.15.52:0/2543412487 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0018070 con 0x7fec000cccd0 2024-05-04T12:19:01.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febfcff9640 1 -- 172.21.15.52:0/2543412487 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf00074e0 con 0x7fec000cccd0 2024-05-04T12:19:01.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7fec05dc4640 1 -- 172.21.15.52:0/2543412487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec00072160 con 0x7fec000cccd0 2024-05-04T12:19:01.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febfcff9640 1 -- 172.21.15.52:0/2543412487 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0004a10 con 0x7fec000cccd0 2024-05-04T12:19:01.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7fec05dc4640 1 -- 172.21.15.52:0/2543412487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec000b0820 con 0x7fec000cccd0 2024-05-04T12:19:01.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7fec05dc4640 1 -- 172.21.15.52:0/2543412487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec00062e50 con 0x7fec000cccd0 2024-05-04T12:19:01.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febfcff9640 1 -- 172.21.15.52:0/2543412487 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7febf001b070 con 0x7fec000cccd0 2024-05-04T12:19:01.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febfcff9640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7febe00678c0 0x7febe0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febfcff9640 1 -- 172.21.15.52:0/2543412487 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7febf0084fa0 con 0x7fec000cccd0 2024-05-04T12:19:01.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febf6ffd640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7febe00678c0 0x7febe0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.033+0000 7febf6ffd640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7febe00678c0 0x7febe0069d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fec00166ee0 tx=0x7febe8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:01.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.037+0000 7febfcff9640 1 -- 172.21.15.52:0/2543412487 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7febf0055b70 con 0x7fec000cccd0 2024-05-04T12:19:01.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.201+0000 7fec05dc4640 1 -- 172.21.15.52:0/2543412487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec00075320 con 0x7fec000cccd0 2024-05-04T12:19:01.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.201+0000 7febfcff9640 1 -- 172.21.15.52:0/2543412487 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7febf0055470 con 0x7fec000cccd0 2024-05-04T12:19:01.205 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:19:01.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.205+0000 7fec05dc4640 1 -- 172.21.15.52:0/2543412487 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7febe00678c0 msgr2=0x7febe0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.205+0000 7fec05dc4640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7febe00678c0 0x7febe0069d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fec00166ee0 tx=0x7febe8008040 comp rx=0 tx=0).stop 2024-05-04T12:19:01.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.205+0000 7fec05dc4640 1 -- 172.21.15.52:0/2543412487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec000cccd0 msgr2=0x7fec00071780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.205+0000 7fec05dc4640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec000cccd0 0x7fec00071780 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7febf00011f0 tx=0x7febf000c9b0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.205+0000 7fec05dc4640 1 -- 172.21.15.52:0/2543412487 shutdown_connections 2024-05-04T12:19:01.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.205+0000 7fec05dc4640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7febe00678c0 0x7febe0069d80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.205+0000 7fec05dc4640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec0015c590 0x7fec0015b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.205+0000 7fec05dc4640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec00145960 0x7fec00144bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.205+0000 7fec05dc4640 1 --2- 172.21.15.52:0/2543412487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec000cccd0 0x7fec00071780 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.205+0000 7fec05dc4640 1 -- 172.21.15.52:0/2543412487 >> 172.21.15.52:0/2543412487 conn(0x7fec00067730 msgr2=0x7fec0015c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:01.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.205+0000 7fec05dc4640 1 -- 172.21.15.52:0/2543412487 shutdown_connections 2024-05-04T12:19:01.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.205+0000 7fec05dc4640 1 -- 172.21.15.52:0/2543412487 wait complete. 2024-05-04T12:19:01.224 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:19:01.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f3940640 1 Processor -- start 2024-05-04T12:19:01.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f3940640 1 -- start start 2024-05-04T12:19:01.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f3940640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff6ec143b80 0x7ff6ec144000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f3940640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6ec06c6c0 0x7ff6ec06cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f3940640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6ec15ed70 con 0x7ff6ec141560 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f3940640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7ff6ec071ed0 con 0x7ff6ec143b80 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f3940640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ec144730 con 0x7ff6ec06c6c0 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f16dc640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7ff6ec141560 0x7ff6ec072360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49956/0 (socket says 172.21.15.52:49956) 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f16dc640 1 -- 172.21.15.52:0/1075774188 learned_addr learned my addr 172.21.15.52:0/1075774188 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f1edd640 1 --2- 172.21.15.52:0/1075774188 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6ec06c6c0 0x7ff6ec06cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f0edb640 1 --2- 172.21.15.52:0/1075774188 >> v2:172.21.15.111:3300/0 conn(0x7ff6ec143b80 0x7ff6ec144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 -- 172.21.15.52:0/1075774188 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2897842071 0 0) 0x7ff6ec15ed70 con 0x7ff6ec141560 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 -- 172.21.15.52:0/1075774188 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6d8003530 con 0x7ff6ec141560 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 -- 172.21.15.52:0/1075774188 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2403702234 0 0) 0x7ff6d8003530 con 0x7ff6ec141560 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 -- 172.21.15.52:0/1075774188 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6ec06c6c0 msgr2=0x7ff6ec06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 --2- 172.21.15.52:0/1075774188 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6ec06c6c0 0x7ff6ec06cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 -- 172.21.15.52:0/1075774188 >> v2:172.21.15.111:3300/0 conn(0x7ff6ec143b80 msgr2=0x7ff6ec144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 --2- 172.21.15.52:0/1075774188 >> v2:172.21.15.111:3300/0 conn(0x7ff6ec143b80 0x7ff6ec144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 -- 172.21.15.52:0/1075774188 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ec071880 con 0x7ff6ec141560 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 -- 172.21.15.52:0/1075774188 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7ff6e8001680 con 0x7ff6ec141560 2024-05-04T12:19:01.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 -- 172.21.15.52:0/1075774188 >> v1:172.21.15.52:6789/0 conn(0x7ff6ec141560 legacy=0x7ff6ec072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 --2- 172.21.15.52:0/1075774188 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6ec06c6c0 0x7ff6d80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 --2- 172.21.15.52:0/1075774188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6d8003f90 0x7ff6d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 --2- 172.21.15.52:0/1075774188 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6d8004d50 0x7ff6d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 -- 172.21.15.52:0/1075774188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec071ed0 con 0x7ff6d8003f90 2024-05-04T12:19:01.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 -- 172.21.15.52:0/1075774188 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec144730 con 0x7ff6d8004d50 2024-05-04T12:19:01.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6e3fff640 1 -- 172.21.15.52:0/1075774188 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d800b9d0 con 0x7ff6ec06c6c0 2024-05-04T12:19:01.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f0edb640 1 --2- 172.21.15.52:0/1075774188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6d8003f90 0x7ff6d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f1edd640 1 --2- 172.21.15.52:0/1075774188 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6d8004d50 0x7ff6d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f16dc640 1 --2- 172.21.15.52:0/1075774188 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6ec06c6c0 0x7ff6d80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f0edb640 1 -- 172.21.15.52:0/1075774188 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6ec06c6c0 msgr2=0x7ff6d80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:01.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f0edb640 1 --2- 172.21.15.52:0/1075774188 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6ec06c6c0 0x7ff6d80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f0edb640 1 -- 172.21.15.52:0/1075774188 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6d8004d50 msgr2=0x7ff6d80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:01.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f0edb640 1 --2- 172.21.15.52:0/1075774188 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6d8004d50 0x7ff6d80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.345+0000 7ff6f0edb640 1 -- 172.21.15.52:0/1075774188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d8003530 con 0x7ff6d8003f90 2024-05-04T12:19:01.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.349+0000 7ff6f0edb640 1 --2- 172.21.15.52:0/1075774188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6d8003f90 0x7ff6d8004810 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009b10 tx=0x7ff6dc01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4463ccd6d6e368b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:01.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.349+0000 7ff6e3fff640 1 -- 172.21.15.52:0/1075774188 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0b2070 con 0x7ff6d8003f90 2024-05-04T12:19:01.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.349+0000 7ff6e3fff640 1 -- 172.21.15.52:0/1075774188 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6dc0a5590 con 0x7ff6d8003f90 2024-05-04T12:19:01.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.349+0000 7ff6e3fff640 1 -- 172.21.15.52:0/1075774188 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0096b0 con 0x7ff6d8003f90 2024-05-04T12:19:01.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.349+0000 7ff6f3940640 1 -- 172.21.15.52:0/1075774188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6d8003f90 msgr2=0x7ff6d8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.349+0000 7ff6f3940640 1 --2- 172.21.15.52:0/1075774188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6d8003f90 0x7ff6d8004810 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009b10 tx=0x7ff6dc01cab0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.349+0000 7ff6f1edd640 1 -- 172.21.15.52:0/1075774188 reap_dead start 2024-05-04T12:19:01.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.349+0000 7ff6f3940640 1 -- 172.21.15.52:0/1075774188 shutdown_connections 2024-05-04T12:19:01.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.349+0000 7ff6f3940640 1 -- 172.21.15.52:0/1075774188 >> 172.21.15.52:0/1075774188 conn(0x7ff6ec06b210 msgr2=0x7ff6ec06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:01.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.349+0000 7ff6f3940640 1 -- 172.21.15.52:0/1075774188 shutdown_connections 2024-05-04T12:19:01.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.349+0000 7ff6f3940640 1 -- 172.21.15.52:0/1075774188 wait complete. 2024-05-04T12:19:01.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.349+0000 7ff6f3940640 1 Processor -- start 2024-05-04T12:19:01.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.349+0000 7ff6f3940640 1 -- start start 2024-05-04T12:19:01.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f3940640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6d8003f90 0x7ff6ec154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f3940640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6ec0b24f0 0x7ff6ec161e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f3940640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6ec0b2970 0x7ff6ec162390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f3940640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec144a10 con 0x7ff6ec0b24f0 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f3940640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec072a70 con 0x7ff6d8003f90 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f3940640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ec06cb50 con 0x7ff6ec0b2970 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f0edb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6ec0b24f0 0x7ff6ec161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f0edb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6ec0b24f0 0x7ff6ec161e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37738/0 (socket says 172.21.15.52:37738) 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f0edb640 1 -- 172.21.15.52:0/2383644709 learned_addr learned my addr 172.21.15.52:0/2383644709 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f16dc640 1 --2- 172.21.15.52:0/2383644709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6d8003f90 0x7ff6ec154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f1edd640 1 --2- 172.21.15.52:0/2383644709 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6ec0b2970 0x7ff6ec162390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f0edb640 1 -- 172.21.15.52:0/2383644709 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6ec0b2970 msgr2=0x7ff6ec162390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f0edb640 1 --2- 172.21.15.52:0/2383644709 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6ec0b2970 0x7ff6ec162390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f0edb640 1 -- 172.21.15.52:0/2383644709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6d8003f90 msgr2=0x7ff6ec154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f0edb640 1 --2- 172.21.15.52:0/2383644709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6d8003f90 0x7ff6ec154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f0edb640 1 -- 172.21.15.52:0/2383644709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ec13dca0 con 0x7ff6ec0b24f0 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f0edb640 1 --2- 172.21.15.52:0/2383644709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6ec0b24f0 0x7ff6ec161e50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009b10 tx=0x7ff6d8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6e1ffb640 1 -- 172.21.15.52:0/2383644709 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0bc030 con 0x7ff6ec0b24f0 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6e1ffb640 1 -- 172.21.15.52:0/2383644709 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6dc00f000 con 0x7ff6ec0b24f0 2024-05-04T12:19:01.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f3940640 1 -- 172.21.15.52:0/2383644709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ec13df80 con 0x7ff6ec0b24f0 2024-05-04T12:19:01.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6e1ffb640 1 -- 172.21.15.52:0/2383644709 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0b2040 con 0x7ff6ec0b24f0 2024-05-04T12:19:01.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f3940640 1 -- 172.21.15.52:0/2383644709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ec13e540 con 0x7ff6ec0b24f0 2024-05-04T12:19:01.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f3940640 1 -- 172.21.15.52:0/2383644709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6ec062e50 con 0x7ff6ec0b24f0 2024-05-04T12:19:01.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6e1ffb640 1 -- 172.21.15.52:0/2383644709 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff6dc0bf070 con 0x7ff6ec0b24f0 2024-05-04T12:19:01.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6e1ffb640 1 --2- 172.21.15.52:0/2383644709 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff6d00678f0 0x7ff6d0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6e1ffb640 1 -- 172.21.15.52:0/2383644709 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff6dc11fdc0 con 0x7ff6ec0b24f0 2024-05-04T12:19:01.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.353+0000 7ff6f16dc640 1 --2- 172.21.15.52:0/2383644709 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff6d00678f0 0x7ff6d0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.357+0000 7ff6f16dc640 1 --2- 172.21.15.52:0/2383644709 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff6d00678f0 0x7ff6d0069db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8001060 tx=0x7ff6e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:01.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.361+0000 7ff6e1ffb640 1 -- 172.21.15.52:0/2383644709 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff6dc016800 con 0x7ff6ec0b24f0 2024-05-04T12:19:01.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.521+0000 7ff6f3940640 1 -- 172.21.15.52:0/2383644709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff6ec075320 con 0x7ff6ec0b24f0 2024-05-04T12:19:01.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.521+0000 7ff6e1ffb640 1 -- 172.21.15.52:0/2383644709 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7ff6dc0f0a10 con 0x7ff6ec0b24f0 2024-05-04T12:19:01.524 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:19:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.521+0000 7ff6f3940640 1 -- 172.21.15.52:0/2383644709 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff6d00678f0 msgr2=0x7ff6d0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.521+0000 7ff6f3940640 1 --2- 172.21.15.52:0/2383644709 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff6d00678f0 0x7ff6d0069db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8001060 tx=0x7ff6e8009040 comp rx=0 tx=0).stop 2024-05-04T12:19:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.521+0000 7ff6f3940640 1 -- 172.21.15.52:0/2383644709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6ec0b24f0 msgr2=0x7ff6ec161e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.521+0000 7ff6f3940640 1 --2- 172.21.15.52:0/2383644709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6ec0b24f0 0x7ff6ec161e50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009b10 tx=0x7ff6d8005840 comp rx=0 tx=0).stop 2024-05-04T12:19:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.521+0000 7ff6f3940640 1 -- 172.21.15.52:0/2383644709 shutdown_connections 2024-05-04T12:19:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.521+0000 7ff6f3940640 1 --2- 172.21.15.52:0/2383644709 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7ff6d00678f0 0x7ff6d0069db0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.521+0000 7ff6f3940640 1 --2- 172.21.15.52:0/2383644709 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6ec0b2970 0x7ff6ec162390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.521+0000 7ff6f3940640 1 --2- 172.21.15.52:0/2383644709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6ec0b24f0 0x7ff6ec161e50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.521+0000 7ff6f3940640 1 --2- 172.21.15.52:0/2383644709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6d8003f90 0x7ff6ec154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.521+0000 7ff6f3940640 1 -- 172.21.15.52:0/2383644709 >> 172.21.15.52:0/2383644709 conn(0x7ff6ec06b210 msgr2=0x7ff6ec15e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.521+0000 7ff6f3940640 1 -- 172.21.15.52:0/2383644709 shutdown_connections 2024-05-04T12:19:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.521+0000 7ff6f3940640 1 -- 172.21.15.52:0/2383644709 wait complete. 2024-05-04T12:19:01.543 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:19:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90b11b0640 1 Processor -- start 2024-05-04T12:19:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90b11b0640 1 -- start start 2024-05-04T12:19:01.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90b11b0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f90ac145960 0x7f90ac160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90b11b0640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f90ac15c590 0x7f90ac0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90b11b0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f90ac0cccd0 0x7f90ac0cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90b11b0640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f90ac0cd140 con 0x7f90ac0cccd0 2024-05-04T12:19:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90b11b0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f90ac06dbf0 con 0x7f90ac145960 2024-05-04T12:19:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90b11b0640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f90ac160ad0 con 0x7f90ac15c590 2024-05-04T12:19:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90ab59e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f90ac0cccd0 0x7f90ac0cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90ab59e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f90ac0cccd0 0x7f90ac0cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37746/0 (socket says 172.21.15.52:37746) 2024-05-04T12:19:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90ab59e640 1 -- 172.21.15.52:0/1681377629 learned_addr learned my addr 172.21.15.52:0/1681377629 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90aa59c640 1 --2- 172.21.15.52:0/1681377629 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f90ac15c590 0x7f90ac0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90aad9d640 1 --2- 172.21.15.52:0/1681377629 >> v2:172.21.15.111:3300/0 conn(0x7f90ac145960 0x7f90ac160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90ab59e640 1 -- 172.21.15.52:0/1681377629 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f90ac15c590 msgr2=0x7f90ac0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90ab59e640 1 --2- 172.21.15.52:0/1681377629 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f90ac15c590 0x7f90ac0cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90ab59e640 1 -- 172.21.15.52:0/1681377629 >> v2:172.21.15.111:3300/0 conn(0x7f90ac145960 msgr2=0x7f90ac160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90ab59e640 1 --2- 172.21.15.52:0/1681377629 >> v2:172.21.15.111:3300/0 conn(0x7f90ac145960 0x7f90ac160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90ab59e640 1 -- 172.21.15.52:0/1681377629 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac06cfe0 con 0x7f90ac0cccd0 2024-05-04T12:19:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.669+0000 7f90aa59c640 1 --2- 172.21.15.52:0/1681377629 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f90ac15c590 0x7f90ac0cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90ab59e640 1 --2- 172.21.15.52:0/1681377629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90ac0cccd0 0x7f90ac0cdf50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f90ac073670 tx=0x7f90a401c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5083ad83a7dbcbb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90a9d9b640 1 -- 172.21.15.52:0/1681377629 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a40b0070 con 0x7f90ac0cccd0 2024-05-04T12:19:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90a9d9b640 1 -- 172.21.15.52:0/1681377629 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a40ab600 con 0x7f90ac0cccd0 2024-05-04T12:19:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90a9d9b640 1 -- 172.21.15.52:0/1681377629 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a40ab9d0 con 0x7f90ac0cccd0 2024-05-04T12:19:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90b11b0640 1 -- 172.21.15.52:0/1681377629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90ac0cccd0 msgr2=0x7f90ac0cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90b11b0640 1 --2- 172.21.15.52:0/1681377629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90ac0cccd0 0x7f90ac0cdf50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f90ac073670 tx=0x7f90a401c1a0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90b11b0640 1 -- 172.21.15.52:0/1681377629 shutdown_connections 2024-05-04T12:19:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90b11b0640 1 --2- 172.21.15.52:0/1681377629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90ac0cccd0 0x7f90ac0cdf50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90b11b0640 1 --2- 172.21.15.52:0/1681377629 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f90ac15c590 0x7f90ac0cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90b11b0640 1 --2- 172.21.15.52:0/1681377629 >> v2:172.21.15.111:3300/0 conn(0x7f90ac145960 0x7f90ac160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90b11b0640 1 -- 172.21.15.52:0/1681377629 >> 172.21.15.52:0/1681377629 conn(0x7f90ac067730 msgr2=0x7f90ac0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90b11b0640 1 -- 172.21.15.52:0/1681377629 shutdown_connections 2024-05-04T12:19:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90b11b0640 1 -- 172.21.15.52:0/1681377629 wait complete. 2024-05-04T12:19:01.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90b11b0640 1 Processor -- start 2024-05-04T12:19:01.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.673+0000 7f90b11b0640 1 -- start start 2024-05-04T12:19:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90b11b0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90ac0cccd0 0x7f90ac15a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90b11b0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90ac145960 0x7f90ac132aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90b11b0640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f90ac15c590 0x7f90ac0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90b11b0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac06cc40 con 0x7f90ac145960 2024-05-04T12:19:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90b11b0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac0cf890 con 0x7f90ac0cccd0 2024-05-04T12:19:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90b11b0640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f90ac06cdc0 con 0x7f90ac15c590 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90aa59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90ac145960 0x7f90ac132aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90aa59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90ac145960 0x7f90ac132aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37748/0 (socket says 172.21.15.52:37748) 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90aa59c640 1 -- 172.21.15.52:0/3250413070 learned_addr learned my addr 172.21.15.52:0/3250413070 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90aad9d640 1 --2- 172.21.15.52:0/3250413070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90ac0cccd0 0x7f90ac15a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90ab59e640 1 --2- 172.21.15.52:0/3250413070 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f90ac15c590 0x7f90ac0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90aa59c640 1 -- 172.21.15.52:0/3250413070 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f90ac15c590 msgr2=0x7f90ac0b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90aa59c640 1 --2- 172.21.15.52:0/3250413070 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f90ac15c590 0x7f90ac0b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90aa59c640 1 -- 172.21.15.52:0/3250413070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90ac0cccd0 msgr2=0x7f90ac15a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90aa59c640 1 --2- 172.21.15.52:0/3250413070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90ac0cccd0 0x7f90ac15a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90aa59c640 1 -- 172.21.15.52:0/3250413070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac1361c0 con 0x7f90ac145960 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90aa59c640 1 --2- 172.21.15.52:0/3250413070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90ac145960 0x7f90ac132aa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f909400e810 tx=0x7f909400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90a37fe640 1 -- 172.21.15.52:0/3250413070 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094017070 con 0x7f90ac145960 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90a37fe640 1 -- 172.21.15.52:0/3250413070 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f909400c8d0 con 0x7f90ac145960 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90b11b0640 1 -- 172.21.15.52:0/3250413070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ac1364a0 con 0x7f90ac145960 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90a37fe640 1 -- 172.21.15.52:0/3250413070 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909400f040 con 0x7f90ac145960 2024-05-04T12:19:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90b11b0640 1 -- 172.21.15.52:0/3250413070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ac136a60 con 0x7f90ac145960 2024-05-04T12:19:01.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90b11b0640 1 -- 172.21.15.52:0/3250413070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90ac062e50 con 0x7f90ac145960 2024-05-04T12:19:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90a37fe640 1 -- 172.21.15.52:0/3250413070 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f909401a070 con 0x7f90ac145960 2024-05-04T12:19:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90a37fe640 1 --2- 172.21.15.52:0/3250413070 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f908c0678c0 0x7f908c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90a37fe640 1 -- 172.21.15.52:0/3250413070 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f90940858c0 con 0x7f90ac145960 2024-05-04T12:19:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90aad9d640 1 --2- 172.21.15.52:0/3250413070 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f908c0678c0 0x7f908c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.677+0000 7f90aad9d640 1 --2- 172.21.15.52:0/3250413070 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f908c0678c0 0x7f908c069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f909c0074a0 tx=0x7f909c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:01.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.681+0000 7f90a37fe640 1 -- 172.21.15.52:0/3250413070 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9094056360 con 0x7f90ac145960 2024-05-04T12:19:01.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.841+0000 7f90b11b0640 1 -- 172.21.15.52:0/3250413070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f90ac075320 con 0x7f90ac145960 2024-05-04T12:19:01.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.845+0000 7f90a37fe640 1 -- 172.21.15.52:0/3250413070 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f9094055c60 con 0x7f90ac145960 2024-05-04T12:19:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:19:01.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.845+0000 7f90b11b0640 1 -- 172.21.15.52:0/3250413070 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f908c0678c0 msgr2=0x7f908c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.845+0000 7f90b11b0640 1 --2- 172.21.15.52:0/3250413070 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f908c0678c0 0x7f908c069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f909c0074a0 tx=0x7f909c00b040 comp rx=0 tx=0).stop 2024-05-04T12:19:01.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.845+0000 7f90b11b0640 1 -- 172.21.15.52:0/3250413070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90ac145960 msgr2=0x7f90ac132aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.845+0000 7f90b11b0640 1 --2- 172.21.15.52:0/3250413070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90ac145960 0x7f90ac132aa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f909400e810 tx=0x7f909400ecd0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.845+0000 7f90b11b0640 1 -- 172.21.15.52:0/3250413070 shutdown_connections 2024-05-04T12:19:01.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.845+0000 7f90b11b0640 1 --2- 172.21.15.52:0/3250413070 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f908c0678c0 0x7f908c069d80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.845+0000 7f90b11b0640 1 --2- 172.21.15.52:0/3250413070 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f90ac15c590 0x7f90ac0b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.845+0000 7f90b11b0640 1 --2- 172.21.15.52:0/3250413070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90ac145960 0x7f90ac132aa0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.845+0000 7f90b11b0640 1 --2- 172.21.15.52:0/3250413070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90ac0cccd0 0x7f90ac15a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.845+0000 7f90b11b0640 1 -- 172.21.15.52:0/3250413070 >> 172.21.15.52:0/3250413070 conn(0x7f90ac067730 msgr2=0x7f90ac15daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:01.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.845+0000 7f90b11b0640 1 -- 172.21.15.52:0/3250413070 shutdown_connections 2024-05-04T12:19:01.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.845+0000 7f90b11b0640 1 -- 172.21.15.52:0/3250413070 wait complete. 2024-05-04T12:19:01.866 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-04T12:19:01.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eaf68f640 1 Processor -- start 2024-05-04T12:19:01.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eaf68f640 1 -- start start 2024-05-04T12:19:01.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eaf68f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1ea815bf50 0x7f1ea815c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eaf68f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ea80aa0c0 0x7f1ea80aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eaf68f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1ea80ccb50 0x7f1ea80cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eaf68f640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea806d1b0 con 0x7f1ea80ccb50 2024-05-04T12:19:01.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eaf68f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea806d490 con 0x7f1ea815bf50 2024-05-04T12:19:01.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eaf68f640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea806d770 con 0x7f1ea80aa0c0 2024-05-04T12:19:01.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eadc2c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1ea80ccb50 0x7f1ea80cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eadc2c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1ea80ccb50 0x7f1ea80cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37758/0 (socket says 172.21.15.52:37758) 2024-05-04T12:19:01.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eadc2c640 1 -- 172.21.15.52:0/900831054 learned_addr learned my addr 172.21.15.52:0/900831054 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:01.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1ead42b640 1 --2- 172.21.15.52:0/900831054 >> v2:172.21.15.111:3300/0 conn(0x7f1ea815bf50 0x7f1ea815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eacc2a640 1 --2- 172.21.15.52:0/900831054 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ea80aa0c0 0x7f1ea80aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eadc2c640 1 -- 172.21.15.52:0/900831054 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ea80aa0c0 msgr2=0x7f1ea80aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:01.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eadc2c640 1 --2- 172.21.15.52:0/900831054 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ea80aa0c0 0x7f1ea80aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eadc2c640 1 -- 172.21.15.52:0/900831054 >> v2:172.21.15.111:3300/0 conn(0x7f1ea815bf50 msgr2=0x7f1ea815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:01.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eadc2c640 1 --2- 172.21.15.52:0/900831054 >> v2:172.21.15.111:3300/0 conn(0x7f1ea815bf50 0x7f1ea815c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.973+0000 7f1eadc2c640 1 -- 172.21.15.52:0/900831054 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea806da50 con 0x7f1ea80ccb50 2024-05-04T12:19:01.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1eadc2c640 1 --2- 172.21.15.52:0/900831054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ea80ccb50 0x7f1ea80cd000 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8069260 tx=0x7f1ea001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32ef87cb4be3e10 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:01.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1e9ffff640 1 -- 172.21.15.52:0/900831054 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00a5780 con 0x7f1ea80ccb50 2024-05-04T12:19:01.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1e9ffff640 1 -- 172.21.15.52:0/900831054 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea00a5920 con 0x7f1ea80ccb50 2024-05-04T12:19:01.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1e9ffff640 1 -- 172.21.15.52:0/900831054 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00a5cf0 con 0x7f1ea80ccb50 2024-05-04T12:19:01.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1eaf68f640 1 -- 172.21.15.52:0/900831054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ea80ccb50 msgr2=0x7f1ea80cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1eaf68f640 1 --2- 172.21.15.52:0/900831054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ea80ccb50 0x7f1ea80cd000 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8069260 tx=0x7f1ea001c9f0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1eaf68f640 1 -- 172.21.15.52:0/900831054 shutdown_connections 2024-05-04T12:19:01.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1eaf68f640 1 --2- 172.21.15.52:0/900831054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ea80ccb50 0x7f1ea80cd000 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1eaf68f640 1 --2- 172.21.15.52:0/900831054 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ea80aa0c0 0x7f1ea80aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1eaf68f640 1 --2- 172.21.15.52:0/900831054 >> v2:172.21.15.111:3300/0 conn(0x7f1ea815bf50 0x7f1ea815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1eaf68f640 1 -- 172.21.15.52:0/900831054 >> 172.21.15.52:0/900831054 conn(0x7f1ea80b14a0 msgr2=0x7f1ea80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:01.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1eaf68f640 1 -- 172.21.15.52:0/900831054 shutdown_connections 2024-05-04T12:19:01.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1eaf68f640 1 -- 172.21.15.52:0/900831054 wait complete. 2024-05-04T12:19:01.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1eaf68f640 1 Processor -- start 2024-05-04T12:19:01.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.977+0000 7f1eaf68f640 1 -- start start 2024-05-04T12:19:01.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eaf68f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ea80aa0c0 0x7f1ea8147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eaf68f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ea80ccb50 0x7f1ea80b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eaf68f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ea815bf50 0x7f1ea80b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eaf68f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea806ea60 con 0x7f1ea80ccb50 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eaf68f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea806cf70 con 0x7f1ea815bf50 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eaf68f640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea806ebe0 con 0x7f1ea80aa0c0 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eacc2a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ea80ccb50 0x7f1ea80b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eacc2a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ea80ccb50 0x7f1ea80b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37768/0 (socket says 172.21.15.52:37768) 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eacc2a640 1 -- 172.21.15.52:0/2100436980 learned_addr learned my addr 172.21.15.52:0/2100436980 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eadc2c640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ea815bf50 0x7f1ea80b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1ead42b640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ea80aa0c0 0x7f1ea8147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eacc2a640 1 -- 172.21.15.52:0/2100436980 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ea80aa0c0 msgr2=0x7f1ea8147dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eacc2a640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ea80aa0c0 0x7f1ea8147dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eacc2a640 1 -- 172.21.15.52:0/2100436980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ea815bf50 msgr2=0x7f1ea80b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eacc2a640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ea815bf50 0x7f1ea80b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eacc2a640 1 -- 172.21.15.52:0/2100436980 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea8071980 con 0x7f1ea80ccb50 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1ead42b640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ea80aa0c0 0x7f1ea8147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eadc2c640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ea815bf50 0x7f1ea80b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:01.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eacc2a640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ea80ccb50 0x7f1ea80b00d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800d800 tx=0x7f1e9800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:01.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1e9dffb640 1 -- 172.21.15.52:0/2100436980 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98018020 con 0x7f1ea80ccb50 2024-05-04T12:19:01.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1e9dffb640 1 -- 172.21.15.52:0/2100436980 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e98009640 con 0x7f1ea80ccb50 2024-05-04T12:19:01.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1e9dffb640 1 -- 172.21.15.52:0/2100436980 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e980099f0 con 0x7f1ea80ccb50 2024-05-04T12:19:01.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eaf68f640 1 -- 172.21.15.52:0/2100436980 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea8071c60 con 0x7f1ea80ccb50 2024-05-04T12:19:01.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eaf68f640 1 -- 172.21.15.52:0/2100436980 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea8072220 con 0x7f1ea80ccb50 2024-05-04T12:19:01.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1eaf68f640 1 -- 172.21.15.52:0/2100436980 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea8062e50 con 0x7f1ea80ccb50 2024-05-04T12:19:01.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1e9dffb640 1 -- 172.21.15.52:0/2100436980 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1e9801b020 con 0x7f1ea80ccb50 2024-05-04T12:19:01.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1e9dffb640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1e8c0678c0 0x7f1e8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:01.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1ead42b640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1e8c0678c0 0x7f1e8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:01.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1e9dffb640 1 -- 172.21.15.52:0/2100436980 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1e98086170 con 0x7f1ea80ccb50 2024-05-04T12:19:01.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.981+0000 7f1ead42b640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1e8c0678c0 0x7f1e8c069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009a40 tx=0x7f1ea4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:01.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:01.985+0000 7f1e9dffb640 1 -- 172.21.15.52:0/2100436980 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1e98022020 con 0x7f1ea80ccb50 2024-05-04T12:19:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.161+0000 7f1eaf68f640 1 -- 172.21.15.52:0/2100436980 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1ea8075320 con 0x7f1ea80ccb50 2024-05-04T12:19:02.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.161+0000 7f1e9dffb640 1 -- 172.21.15.52:0/2100436980 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f1e98056d40 con 0x7f1ea80ccb50 2024-05-04T12:19:02.166 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-04T12:19:02.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.165+0000 7f1eaf68f640 1 -- 172.21.15.52:0/2100436980 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1e8c0678c0 msgr2=0x7f1e8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:02.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.165+0000 7f1eaf68f640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1e8c0678c0 0x7f1e8c069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009a40 tx=0x7f1ea4008040 comp rx=0 tx=0).stop 2024-05-04T12:19:02.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.165+0000 7f1eaf68f640 1 -- 172.21.15.52:0/2100436980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ea80ccb50 msgr2=0x7f1ea80b00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:02.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.165+0000 7f1eaf68f640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ea80ccb50 0x7f1ea80b00d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800d800 tx=0x7f1e9800dcc0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.165+0000 7f1eaf68f640 1 -- 172.21.15.52:0/2100436980 shutdown_connections 2024-05-04T12:19:02.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.165+0000 7f1eaf68f640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1e8c0678c0 0x7f1e8c069d80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.165+0000 7f1eaf68f640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ea815bf50 0x7f1ea80b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.165+0000 7f1eaf68f640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ea80ccb50 0x7f1ea80b00d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.165+0000 7f1eaf68f640 1 --2- 172.21.15.52:0/2100436980 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ea80aa0c0 0x7f1ea8147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.165+0000 7f1eaf68f640 1 -- 172.21.15.52:0/2100436980 >> 172.21.15.52:0/2100436980 conn(0x7f1ea80b14a0 msgr2=0x7f1ea81592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:02.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.165+0000 7f1eaf68f640 1 -- 172.21.15.52:0/2100436980 shutdown_connections 2024-05-04T12:19:02.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.165+0000 7f1eaf68f640 1 -- 172.21.15.52:0/2100436980 wait complete. 2024-05-04T12:19:02.185 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2024-05-04T12:19:02.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:02.185 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-05-04T12:19:02.214 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T12:19:02.214 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T12:19:02.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:02.214 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-04T12:19:02.270 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:19:02.270 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T12:19:02.276 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:19:02.276 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T12:19:02.323 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:19:02.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.485+0000 7fccd8991640 1 Processor -- start 2024-05-04T12:19:02.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.485+0000 7fccd8991640 1 -- start start 2024-05-04T12:19:02.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.485+0000 7fccd8991640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fccd40aa0c0 0x7fccd40ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:02.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.485+0000 7fccd8991640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccd406d1b0 con 0x7fccd415bf50 2024-05-04T12:19:02.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.485+0000 7fccd8991640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccd406d490 con 0x7fccd40aa550 2024-05-04T12:19:02.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.485+0000 7fccd8991640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fccd415ca40 con 0x7fccd40aa0c0 2024-05-04T12:19:02.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.485+0000 7fccd259c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fccd415bf50 0x7fccd415c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49964/0 (socket says 172.21.15.52:49964) 2024-05-04T12:19:02.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.485+0000 7fccd259c640 1 -- 172.21.15.52:0/4235116517 learned_addr learned my addr 172.21.15.52:0/4235116517 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:02.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.485+0000 7fccd1d9b640 1 --2- 172.21.15.52:0/4235116517 >> v2:172.21.15.52:3300/0 conn(0x7fccd40aa0c0 0x7fccd40ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:02.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.485+0000 7fccd159a640 1 -- 172.21.15.52:0/4235116517 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3317401269 0 0) 0x7fccd406d1b0 con 0x7fccd415bf50 2024-05-04T12:19:02.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.485+0000 7fccd159a640 1 -- 172.21.15.52:0/4235116517 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccc0003530 con 0x7fccd415bf50 2024-05-04T12:19:02.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.485+0000 7fccd1d9b640 1 -- 172.21.15.52:0/4235116517 >> v1:172.21.15.111:6789/0 conn(0x7fccd40aa550 legacy=0x7fccd406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:02.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.485+0000 7fccd1d9b640 1 -- 172.21.15.52:0/4235116517 >> v1:172.21.15.52:6789/0 conn(0x7fccd415bf50 legacy=0x7fccd415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:02.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.485+0000 7fccd1d9b640 1 -- 172.21.15.52:0/4235116517 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd406d770 con 0x7fccd40aa0c0 2024-05-04T12:19:02.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.489+0000 7fccd1d9b640 1 --2- 172.21.15.52:0/4235116517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccd40aa0c0 0x7fccd40ccb50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0097c0 tx=0x7fccbc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd2861ab97db6c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.489+0000 7fccd159a640 1 -- 172.21.15.52:0/4235116517 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc0b0070 con 0x7fccd40aa0c0 2024-05-04T12:19:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.489+0000 7fccd159a640 1 -- 172.21.15.52:0/4235116517 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccbc0ab600 con 0x7fccd40aa0c0 2024-05-04T12:19:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.489+0000 7fccd159a640 1 -- 172.21.15.52:0/4235116517 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc0ab9d0 con 0x7fccd40aa0c0 2024-05-04T12:19:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.489+0000 7fccd8991640 1 -- 172.21.15.52:0/4235116517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccd40aa0c0 msgr2=0x7fccd40ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.489+0000 7fccd8991640 1 --2- 172.21.15.52:0/4235116517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccd40aa0c0 0x7fccd40ccb50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0097c0 tx=0x7fccbc01c430 comp rx=0 tx=0).stop 2024-05-04T12:19:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.489+0000 7fccd8991640 1 -- 172.21.15.52:0/4235116517 shutdown_connections 2024-05-04T12:19:02.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.489+0000 7fccd8991640 1 --2- 172.21.15.52:0/4235116517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccd40aa0c0 0x7fccd40ccb50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.489+0000 7fccd8991640 1 -- 172.21.15.52:0/4235116517 >> 172.21.15.52:0/4235116517 conn(0x7fccd40b14a0 msgr2=0x7fccd40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:02.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.489+0000 7fccd8991640 1 -- 172.21.15.52:0/4235116517 shutdown_connections 2024-05-04T12:19:02.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.489+0000 7fccd8991640 1 -- 172.21.15.52:0/4235116517 wait complete. 2024-05-04T12:19:02.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.489+0000 7fccd8991640 1 Processor -- start 2024-05-04T12:19:02.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.489+0000 7fccd8991640 1 -- start start 2024-05-04T12:19:02.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd8991640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccd40aa0c0 0x7fccd41fa750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:02.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd8991640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd40aa550 0x7fccd40714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:02.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd8991640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccd415bf50 0x7fccd40b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:02.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd8991640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fccd416a040 con 0x7fccd415bf50 2024-05-04T12:19:02.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd8991640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fccd406cf70 con 0x7fccd40aa550 2024-05-04T12:19:02.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd8991640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fccd416a1c0 con 0x7fccd40aa0c0 2024-05-04T12:19:02.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd2d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccd415bf50 0x7fccd40b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd2d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccd415bf50 0x7fccd40b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37784/0 (socket says 172.21.15.52:37784) 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd2d9d640 1 -- 172.21.15.52:0/202885558 learned_addr learned my addr 172.21.15.52:0/202885558 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd259c640 1 --2- 172.21.15.52:0/202885558 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccd40aa0c0 0x7fccd41fa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd1d9b640 1 --2- 172.21.15.52:0/202885558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd40aa550 0x7fccd40714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd2d9d640 1 -- 172.21.15.52:0/202885558 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccd40aa0c0 msgr2=0x7fccd41fa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd2d9d640 1 --2- 172.21.15.52:0/202885558 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccd40aa0c0 0x7fccd41fa750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd2d9d640 1 -- 172.21.15.52:0/202885558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd40aa550 msgr2=0x7fccd40714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd2d9d640 1 --2- 172.21.15.52:0/202885558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd40aa550 0x7fccd40714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd2d9d640 1 -- 172.21.15.52:0/202885558 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd41faf70 con 0x7fccd415bf50 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd2d9d640 1 --2- 172.21.15.52:0/202885558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccd415bf50 0x7fccd40b01a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00c780 tx=0x7fcccc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fcccaffd640 1 -- 172.21.15.52:0/202885558 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc004b70 con 0x7fccd415bf50 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fcccaffd640 1 -- 172.21.15.52:0/202885558 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcccc004d10 con 0x7fccd415bf50 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd8991640 1 -- 172.21.15.52:0/202885558 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd41fb250 con 0x7fccd415bf50 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fcccaffd640 1 -- 172.21.15.52:0/202885558 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc00aa90 con 0x7fccd415bf50 2024-05-04T12:19:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd8991640 1 -- 172.21.15.52:0/202885558 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd41fb810 con 0x7fccd415bf50 2024-05-04T12:19:02.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd8991640 1 -- 172.21.15.52:0/202885558 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd4062e50 con 0x7fccd415bf50 2024-05-04T12:19:02.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fcccaffd640 1 -- 172.21.15.52:0/202885558 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcccc00ac30 con 0x7fccd415bf50 2024-05-04T12:19:02.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fcccaffd640 1 --2- 172.21.15.52:0/202885558 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fccb40678c0 0x7fccb4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:02.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fcccaffd640 1 -- 172.21.15.52:0/202885558 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcccc085ff0 con 0x7fccd415bf50 2024-05-04T12:19:02.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd259c640 1 --2- 172.21.15.52:0/202885558 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fccb40678c0 0x7fccb4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:02.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.493+0000 7fccd259c640 1 --2- 172.21.15.52:0/202885558 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fccb40678c0 0x7fccb4069d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fccc4005890 tx=0x7fccc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:02.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.497+0000 7fcccaffd640 1 -- 172.21.15.52:0/202885558 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcccc020070 con 0x7fccd415bf50 2024-05-04T12:19:02.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.653+0000 7fccd8991640 1 -- 172.21.15.52:0/202885558 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccd4075320 con 0x7fccd415bf50 2024-05-04T12:19:02.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.657+0000 7fcccaffd640 1 -- 172.21.15.52:0/202885558 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fcccc056ab0 con 0x7fccd415bf50 2024-05-04T12:19:02.661 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:02.661 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":21,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:18:23.825538+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:19:02.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.657+0000 7fccd8991640 1 -- 172.21.15.52:0/202885558 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fccb40678c0 msgr2=0x7fccb4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:02.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.657+0000 7fccd8991640 1 --2- 172.21.15.52:0/202885558 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fccb40678c0 0x7fccb4069d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fccc4005890 tx=0x7fccc4009040 comp rx=0 tx=0).stop 2024-05-04T12:19:02.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.657+0000 7fccd8991640 1 -- 172.21.15.52:0/202885558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccd415bf50 msgr2=0x7fccd40b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:02.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.657+0000 7fccd8991640 1 --2- 172.21.15.52:0/202885558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccd415bf50 0x7fccd40b01a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00c780 tx=0x7fcccc00cc40 comp rx=0 tx=0).stop 2024-05-04T12:19:02.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.657+0000 7fccd8991640 1 -- 172.21.15.52:0/202885558 shutdown_connections 2024-05-04T12:19:02.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.657+0000 7fccd8991640 1 --2- 172.21.15.52:0/202885558 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fccb40678c0 0x7fccb4069d80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.657+0000 7fccd8991640 1 --2- 172.21.15.52:0/202885558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccd415bf50 0x7fccd40b01a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.657+0000 7fccd8991640 1 --2- 172.21.15.52:0/202885558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd40aa550 0x7fccd40714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.657+0000 7fccd8991640 1 --2- 172.21.15.52:0/202885558 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccd40aa0c0 0x7fccd41fa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.657+0000 7fccd8991640 1 -- 172.21.15.52:0/202885558 >> 172.21.15.52:0/202885558 conn(0x7fccd40b14a0 msgr2=0x7fccd414b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:02.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.657+0000 7fccd8991640 1 -- 172.21.15.52:0/202885558 shutdown_connections 2024-05-04T12:19:02.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.661+0000 7fccd8991640 1 -- 172.21.15.52:0/202885558 wait complete. 2024-05-04T12:19:02.678 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:19:02.679 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:19:02.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f307f1640 1 Processor -- start 2024-05-04T12:19:02.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f307f1640 1 -- start start 2024-05-04T12:19:02.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f307f1640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f280cf770 0x7f1f2806f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:02.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f307f1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1f2815c170 0x7f1f2806f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:02.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f307f1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f280ccb80 con 0x7f1f2815bd90 2024-05-04T12:19:02.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f307f1640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f1f2806fdf0 con 0x7f1f2815c170 2024-05-04T12:19:02.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f307f1640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f1f280cc930 con 0x7f1f280cf770 2024-05-04T12:19:02.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f2e58d640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f1f2815bd90 0x7f1f280cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49968/0 (socket says 172.21.15.52:49968) 2024-05-04T12:19:02.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f2e58d640 1 -- 172.21.15.52:0/3842638365 learned_addr learned my addr 172.21.15.52:0/3842638365 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:02.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f2ed8e640 1 --2- 172.21.15.52:0/3842638365 >> v2:172.21.15.52:3300/0 conn(0x7f1f2815c170 0x7f1f2806f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:02.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f2dd8c640 1 --2- 172.21.15.52:0/3842638365 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f280cf770 0x7f1f2806f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:02.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f2d58b640 1 -- 172.21.15.52:0/3842638365 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 330420149 0 0) 0x7f1f280ccb80 con 0x7f1f2815bd90 2024-05-04T12:19:02.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f2d58b640 1 -- 172.21.15.52:0/3842638365 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f14003530 con 0x7f1f2815bd90 2024-05-04T12:19:02.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f2ed8e640 1 -- 172.21.15.52:0/3842638365 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f280cf770 msgr2=0x7f1f2806f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:02.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f2ed8e640 1 --2- 172.21.15.52:0/3842638365 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f280cf770 0x7f1f2806f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f2ed8e640 1 -- 172.21.15.52:0/3842638365 >> v1:172.21.15.52:6789/0 conn(0x7f1f2815bd90 legacy=0x7f1f280cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:02.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f2ed8e640 1 -- 172.21.15.52:0/3842638365 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f280cce60 con 0x7f1f2815c170 2024-05-04T12:19:02.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.805+0000 7f1f2dd8c640 1 --2- 172.21.15.52:0/3842638365 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f280cf770 0x7f1f2806f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2ed8e640 1 --2- 172.21.15.52:0/3842638365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f2815c170 0x7f1f2806f8b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009d90 tx=0x7f1f2001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1346c0d747225a38 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2d58b640 1 -- 172.21.15.52:0/3842638365 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f200b2070 con 0x7f1f2815c170 2024-05-04T12:19:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2d58b640 1 -- 172.21.15.52:0/3842638365 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f200a5830 con 0x7f1f2815c170 2024-05-04T12:19:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2d58b640 1 -- 172.21.15.52:0/3842638365 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f200a5c00 con 0x7f1f2815c170 2024-05-04T12:19:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 -- 172.21.15.52:0/3842638365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f2815c170 msgr2=0x7f1f2806f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:02.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 --2- 172.21.15.52:0/3842638365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f2815c170 0x7f1f2806f8b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009d90 tx=0x7f1f2001c8e0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 -- 172.21.15.52:0/3842638365 shutdown_connections 2024-05-04T12:19:02.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 --2- 172.21.15.52:0/3842638365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f2815c170 0x7f1f2806f8b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 --2- 172.21.15.52:0/3842638365 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f280cf770 0x7f1f2806f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 -- 172.21.15.52:0/3842638365 >> 172.21.15.52:0/3842638365 conn(0x7f1f28067730 msgr2=0x7f1f280b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:02.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 -- 172.21.15.52:0/3842638365 shutdown_connections 2024-05-04T12:19:02.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 -- 172.21.15.52:0/3842638365 wait complete. 2024-05-04T12:19:02.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 Processor -- start 2024-05-04T12:19:02.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 -- start start 2024-05-04T12:19:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f280cf770 0x7f1f2814ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f2815bd90 0x7f1f2814e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f2815c170 0x7f1f2815c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2806d7e0 con 0x7f1f280cf770 2024-05-04T12:19:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2806d660 con 0x7f1f2815bd90 2024-05-04T12:19:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f1f2806d960 con 0x7f1f2815c170 2024-05-04T12:19:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2e58d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f280cf770 0x7f1f2814ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2e58d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f280cf770 0x7f1f2814ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37800/0 (socket says 172.21.15.52:37800) 2024-05-04T12:19:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2e58d640 1 -- 172.21.15.52:0/1760889597 learned_addr learned my addr 172.21.15.52:0/1760889597 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2dd8c640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f2815bd90 0x7f1f2814e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2ed8e640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f2815c170 0x7f1f2815c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2e58d640 1 -- 172.21.15.52:0/1760889597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f2815c170 msgr2=0x7f1f2815c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2e58d640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f2815c170 0x7f1f2815c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2e58d640 1 -- 172.21.15.52:0/1760889597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f2815bd90 msgr2=0x7f1f2814e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2e58d640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f2815bd90 0x7f1f2814e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2e58d640 1 -- 172.21.15.52:0/1760889597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2814e770 con 0x7f1f280cf770 2024-05-04T12:19:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2ed8e640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f2815c170 0x7f1f2815c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2dd8c640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f2815bd90 0x7f1f2814e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f2e58d640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f280cf770 0x7f1f2814ba70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400c3c0 tx=0x7f1f2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f1effd640 1 -- 172.21.15.52:0/1760889597 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24017070 con 0x7f1f280cf770 2024-05-04T12:19:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f1effd640 1 -- 172.21.15.52:0/1760889597 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f24005cf0 con 0x7f1f280cf770 2024-05-04T12:19:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 -- 172.21.15.52:0/1760889597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2814ea10 con 0x7f1f280cf770 2024-05-04T12:19:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f1effd640 1 -- 172.21.15.52:0/1760889597 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f240072f0 con 0x7f1f280cf770 2024-05-04T12:19:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.809+0000 7f1f307f1640 1 -- 172.21.15.52:0/1760889597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f2814efd0 con 0x7f1f280cf770 2024-05-04T12:19:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.813+0000 7f1f307f1640 1 -- 172.21.15.52:0/1760889597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f28062e50 con 0x7f1f280cf770 2024-05-04T12:19:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.813+0000 7f1f1effd640 1 -- 172.21.15.52:0/1760889597 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1f2401a070 con 0x7f1f280cf770 2024-05-04T12:19:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.813+0000 7f1f1effd640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1f0c0678c0 0x7f1f0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.813+0000 7f1f1effd640 1 -- 172.21.15.52:0/1760889597 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1f24085040 con 0x7f1f280cf770 2024-05-04T12:19:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.813+0000 7f1f2dd8c640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1f0c0678c0 0x7f1f0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.813+0000 7f1f2dd8c640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1f0c0678c0 0x7f1f0c069d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f2815b030 tx=0x7f1f18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:02.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.817+0000 7f1f1effd640 1 -- 172.21.15.52:0/1760889597 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1f24055b80 con 0x7f1f280cf770 2024-05-04T12:19:02.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.981+0000 7f1f307f1640 1 -- 172.21.15.52:0/1760889597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1f28075320 con 0x7f1f280cf770 2024-05-04T12:19:02.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.981+0000 7f1f1effd640 1 -- 172.21.15.52:0/1760889597 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f1f24055480 con 0x7f1f280cf770 2024-05-04T12:19:02.986 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:19:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.985+0000 7f1f307f1640 1 -- 172.21.15.52:0/1760889597 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1f0c0678c0 msgr2=0x7f1f0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.985+0000 7f1f307f1640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1f0c0678c0 0x7f1f0c069d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f2815b030 tx=0x7f1f18008040 comp rx=0 tx=0).stop 2024-05-04T12:19:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.985+0000 7f1f307f1640 1 -- 172.21.15.52:0/1760889597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f280cf770 msgr2=0x7f1f2814ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.985+0000 7f1f307f1640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f280cf770 0x7f1f2814ba70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400c3c0 tx=0x7f1f2400c880 comp rx=0 tx=0).stop 2024-05-04T12:19:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.985+0000 7f1f307f1640 1 -- 172.21.15.52:0/1760889597 shutdown_connections 2024-05-04T12:19:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.985+0000 7f1f307f1640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1f0c0678c0 0x7f1f0c069d80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.985+0000 7f1f307f1640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f2815c170 0x7f1f2815c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.985+0000 7f1f307f1640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f2815bd90 0x7f1f2814e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.985+0000 7f1f307f1640 1 --2- 172.21.15.52:0/1760889597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f280cf770 0x7f1f2814ba70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.985+0000 7f1f307f1640 1 -- 172.21.15.52:0/1760889597 >> 172.21.15.52:0/1760889597 conn(0x7f1f28067730 msgr2=0x7f1f280cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.985+0000 7f1f307f1640 1 -- 172.21.15.52:0/1760889597 shutdown_connections 2024-05-04T12:19:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:02.985+0000 7f1f307f1640 1 -- 172.21.15.52:0/1760889597 wait complete. 2024-05-04T12:19:03.006 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:19:03.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.129+0000 7fab06004640 1 Processor -- start 2024-05-04T12:19:03.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.129+0000 7fab06004640 1 -- start start 2024-05-04T12:19:03.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7fab06004640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab00145960 0x7fab00160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7fab06004640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fab0015c590 0x7fab000cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7fab06004640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab0006dbf0 con 0x7fab000cccd0 2024-05-04T12:19:03.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7fab06004640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fab000cd140 con 0x7fab0015c590 2024-05-04T12:19:03.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7fab06004640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fab00160ad0 con 0x7fab00145960 2024-05-04T12:19:03.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7faafffff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fab000cccd0 0x7fab000cdf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49970/0 (socket says 172.21.15.52:49970) 2024-05-04T12:19:03.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7faafffff640 1 -- 172.21.15.52:0/1280476522 learned_addr learned my addr 172.21.15.52:0/1280476522 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:03.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7faafeffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fab0015c590 0x7fab000cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7faaff7fe640 1 --2- 172.21.15.52:0/1280476522 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab00145960 0x7fab00160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7faafeffd640 1 -- 172.21.15.52:0/1280476522 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab00145960 msgr2=0x7fab00160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:03.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7faafeffd640 1 --2- 172.21.15.52:0/1280476522 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab00145960 0x7fab00160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7faafeffd640 1 -- 172.21.15.52:0/1280476522 >> v1:172.21.15.52:6789/0 conn(0x7fab000cccd0 legacy=0x7fab000cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7faafe7fc640 1 -- 172.21.15.52:0/1280476522 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4231092249 0 0) 0x7fab0006dbf0 con 0x7fab000cccd0 2024-05-04T12:19:03.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7faafeffd640 1 -- 172.21.15.52:0/1280476522 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0006f090 con 0x7fab0015c590 2024-05-04T12:19:03.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7faafeffd640 1 --2- 172.21.15.52:0/1280476522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab0015c590 0x7fab000cc790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faaf40097c0 tx=0x7faaf401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da52e8bd891db00 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:03.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7faafe7fc640 1 -- 172.21.15.52:0/1280476522 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf40b0070 con 0x7fab0015c590 2024-05-04T12:19:03.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7faafe7fc640 1 -- 172.21.15.52:0/1280476522 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf40ab600 con 0x7fab0015c590 2024-05-04T12:19:03.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7faafe7fc640 1 -- 172.21.15.52:0/1280476522 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf40ab9d0 con 0x7fab0015c590 2024-05-04T12:19:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7fab06004640 1 -- 172.21.15.52:0/1280476522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab0015c590 msgr2=0x7fab000cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.133+0000 7fab06004640 1 --2- 172.21.15.52:0/1280476522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab0015c590 0x7fab000cc790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faaf40097c0 tx=0x7faaf401c3e0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 -- 172.21.15.52:0/1280476522 shutdown_connections 2024-05-04T12:19:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 --2- 172.21.15.52:0/1280476522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab0015c590 0x7fab000cc790 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 --2- 172.21.15.52:0/1280476522 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab00145960 0x7fab00160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 -- 172.21.15.52:0/1280476522 >> 172.21.15.52:0/1280476522 conn(0x7fab00067730 msgr2=0x7fab000b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 -- 172.21.15.52:0/1280476522 shutdown_connections 2024-05-04T12:19:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 -- 172.21.15.52:0/1280476522 wait complete. 2024-05-04T12:19:03.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 Processor -- start 2024-05-04T12:19:03.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 -- start start 2024-05-04T12:19:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab000cccd0 0x7fab001596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab00145960 0x7fab0013cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab0015c590 0x7fab0013d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fab000cf890 con 0x7fab000cccd0 2024-05-04T12:19:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fab0006d880 con 0x7fab0015c590 2024-05-04T12:19:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fab0006cc40 con 0x7fab00145960 2024-05-04T12:19:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faaff7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab000cccd0 0x7fab001596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faaff7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab000cccd0 0x7fab001596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37806/0 (socket says 172.21.15.52:37806) 2024-05-04T12:19:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faaff7fe640 1 -- 172.21.15.52:0/1363449665 learned_addr learned my addr 172.21.15.52:0/1363449665 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faafffff640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab0015c590 0x7fab0013d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faafeffd640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab00145960 0x7fab0013cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faaff7fe640 1 -- 172.21.15.52:0/1363449665 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab00145960 msgr2=0x7fab0013cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faaff7fe640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab00145960 0x7fab0013cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faaff7fe640 1 -- 172.21.15.52:0/1363449665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab0015c590 msgr2=0x7fab0013d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faaff7fe640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab0015c590 0x7fab0013d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faaff7fe640 1 -- 172.21.15.52:0/1363449665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0013ff40 con 0x7fab000cccd0 2024-05-04T12:19:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faafffff640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab0015c590 0x7fab0013d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faaff7fe640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab000cccd0 0x7fab001596e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faaf000d830 tx=0x7faaf000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faae7fff640 1 -- 172.21.15.52:0/1363449665 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf000b6c0 con 0x7fab000cccd0 2024-05-04T12:19:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faae7fff640 1 -- 172.21.15.52:0/1363449665 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf0019070 con 0x7fab000cccd0 2024-05-04T12:19:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 -- 172.21.15.52:0/1363449665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0013d8b0 con 0x7fab000cccd0 2024-05-04T12:19:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7faae7fff640 1 -- 172.21.15.52:0/1363449665 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf000ba60 con 0x7fab000cccd0 2024-05-04T12:19:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 -- 172.21.15.52:0/1363449665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab00136540 con 0x7fab000cccd0 2024-05-04T12:19:03.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.137+0000 7fab06004640 1 -- 172.21.15.52:0/1363449665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab00062e50 con 0x7fab000cccd0 2024-05-04T12:19:03.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.141+0000 7faae7fff640 1 -- 172.21.15.52:0/1363449665 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faaf000bc00 con 0x7fab000cccd0 2024-05-04T12:19:03.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.141+0000 7faae7fff640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faaec0678f0 0x7faaec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.141+0000 7faae7fff640 1 -- 172.21.15.52:0/1363449665 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7faaf00861c0 con 0x7fab000cccd0 2024-05-04T12:19:03.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.141+0000 7faafeffd640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faaec0678f0 0x7faaec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.141+0000 7faafeffd640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faaec0678f0 0x7faaec069db0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faaf40ad000 tx=0x7faaf40a4f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:03.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.145+0000 7faae7fff640 1 -- 172.21.15.52:0/1363449665 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faaf0056c70 con 0x7fab000cccd0 2024-05-04T12:19:03.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.297+0000 7fab06004640 1 -- 172.21.15.52:0/1363449665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fab0013ddd0 con 0x7fab000cccd0 2024-05-04T12:19:03.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.297+0000 7faae7fff640 1 -- 172.21.15.52:0/1363449665 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7faaf0056570 con 0x7fab000cccd0 2024-05-04T12:19:03.303 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:19:03.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.301+0000 7fab06004640 1 -- 172.21.15.52:0/1363449665 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faaec0678f0 msgr2=0x7faaec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.301+0000 7fab06004640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faaec0678f0 0x7faaec069db0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faaf40ad000 tx=0x7faaf40a4f90 comp rx=0 tx=0).stop 2024-05-04T12:19:03.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.301+0000 7fab06004640 1 -- 172.21.15.52:0/1363449665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab000cccd0 msgr2=0x7fab001596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.301+0000 7fab06004640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab000cccd0 0x7fab001596e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faaf000d830 tx=0x7faaf000dcf0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.301+0000 7fab06004640 1 -- 172.21.15.52:0/1363449665 shutdown_connections 2024-05-04T12:19:03.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.301+0000 7fab06004640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faaec0678f0 0x7faaec069db0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.301+0000 7fab06004640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab0015c590 0x7fab0013d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.301+0000 7fab06004640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab00145960 0x7fab0013cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.301+0000 7fab06004640 1 --2- 172.21.15.52:0/1363449665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab000cccd0 0x7fab001596e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.301+0000 7fab06004640 1 -- 172.21.15.52:0/1363449665 >> 172.21.15.52:0/1363449665 conn(0x7fab00067730 msgr2=0x7fab000b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:03.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.301+0000 7fab06004640 1 -- 172.21.15.52:0/1363449665 shutdown_connections 2024-05-04T12:19:03.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.301+0000 7fab06004640 1 -- 172.21.15.52:0/1363449665 wait complete. 2024-05-04T12:19:03.320 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:19:03.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.433+0000 7f34dea9c640 1 Processor -- start 2024-05-04T12:19:03.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.433+0000 7f34dea9c640 1 -- start start 2024-05-04T12:19:03.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34dea9c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f34d806cdd0 0x7f34d806d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34dea9c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f34d815ea70 0x7f34d80d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34dea9c640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34d8072b90 con 0x7f34d8147280 2024-05-04T12:19:03.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34dea9c640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f34d8147660 con 0x7f34d806cdd0 2024-05-04T12:19:03.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34dea9c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f34d815ef00 con 0x7f34d815ea70 2024-05-04T12:19:03.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34d7fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f34d806cdd0 0x7f34d806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34dc838640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f34d8147280 0x7f34d806c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49984/0 (socket says 172.21.15.52:49984) 2024-05-04T12:19:03.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34dc838640 1 -- 172.21.15.52:0/646281684 learned_addr learned my addr 172.21.15.52:0/646281684 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:03.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34dd039640 1 --2- 172.21.15.52:0/646281684 >> v2:172.21.15.111:3300/0 conn(0x7f34d815ea70 0x7f34d80d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34d77fe640 1 -- 172.21.15.52:0/646281684 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1731694739 0 0) 0x7f34c8001120 con 0x7f34d8147280 2024-05-04T12:19:03.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34d7fff640 1 -- 172.21.15.52:0/646281684 >> v2:172.21.15.111:3300/0 conn(0x7f34d815ea70 msgr2=0x7f34d80d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:03.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34d7fff640 1 --2- 172.21.15.52:0/646281684 >> v2:172.21.15.111:3300/0 conn(0x7f34d815ea70 0x7f34d80d0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34d7fff640 1 -- 172.21.15.52:0/646281684 >> v1:172.21.15.52:6789/0 conn(0x7f34d8147280 legacy=0x7f34d806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34d7fff640 1 -- 172.21.15.52:0/646281684 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d80725d0 con 0x7f34d806cdd0 2024-05-04T12:19:03.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.437+0000 7f34d7fff640 1 --2- 172.21.15.52:0/646281684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34d806cdd0 0x7f34d806d400 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f34d806ee10 tx=0x7f34cc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179365a73da072c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:03.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34d77fe640 1 -- 172.21.15.52:0/646281684 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc0ab650 con 0x7f34d806cdd0 2024-05-04T12:19:03.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34d77fe640 1 -- 172.21.15.52:0/646281684 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34cc0b0070 con 0x7f34d806cdd0 2024-05-04T12:19:03.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34d77fe640 1 -- 172.21.15.52:0/646281684 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc0aba10 con 0x7f34d806cdd0 2024-05-04T12:19:03.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 -- 172.21.15.52:0/646281684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34d806cdd0 msgr2=0x7f34d806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 --2- 172.21.15.52:0/646281684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34d806cdd0 0x7f34d806d400 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f34d806ee10 tx=0x7f34cc01c450 comp rx=0 tx=0).stop 2024-05-04T12:19:03.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 -- 172.21.15.52:0/646281684 shutdown_connections 2024-05-04T12:19:03.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 --2- 172.21.15.52:0/646281684 >> v2:172.21.15.111:3300/0 conn(0x7f34d815ea70 0x7f34d80d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 --2- 172.21.15.52:0/646281684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34d806cdd0 0x7f34d806d400 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 -- 172.21.15.52:0/646281684 >> 172.21.15.52:0/646281684 conn(0x7f34d806b210 msgr2=0x7f34d806b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:03.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 -- 172.21.15.52:0/646281684 shutdown_connections 2024-05-04T12:19:03.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 -- 172.21.15.52:0/646281684 wait complete. 2024-05-04T12:19:03.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 Processor -- start 2024-05-04T12:19:03.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 -- start start 2024-05-04T12:19:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d806cdd0 0x7f34d80aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f34d8147280 0x7f34d8140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34d815ea70 0x7f34d8140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f34d81480a0 con 0x7f34d815ea70 2024-05-04T12:19:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f34d8147f20 con 0x7f34d806cdd0 2024-05-04T12:19:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dea9c640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f34d80721c0 con 0x7f34d8147280 2024-05-04T12:19:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dd039640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34d815ea70 0x7f34d8140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dd039640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34d815ea70 0x7f34d8140580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37828/0 (socket says 172.21.15.52:37828) 2024-05-04T12:19:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dd039640 1 -- 172.21.15.52:0/1671453946 learned_addr learned my addr 172.21.15.52:0/1671453946 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dc838640 1 --2- 172.21.15.52:0/1671453946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d806cdd0 0x7f34d80aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34d7fff640 1 --2- 172.21.15.52:0/1671453946 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f34d8147280 0x7f34d8140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dd039640 1 -- 172.21.15.52:0/1671453946 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f34d8147280 msgr2=0x7f34d8140040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dd039640 1 --2- 172.21.15.52:0/1671453946 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f34d8147280 0x7f34d8140040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dd039640 1 -- 172.21.15.52:0/1671453946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d806cdd0 msgr2=0x7f34d80aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dd039640 1 --2- 172.21.15.52:0/1671453946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d806cdd0 0x7f34d80aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dd039640 1 -- 172.21.15.52:0/1671453946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d815df90 con 0x7f34d815ea70 2024-05-04T12:19:03.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34dd039640 1 --2- 172.21.15.52:0/1671453946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34d815ea70 0x7f34d8140580 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f34d0007e70 tx=0x7f34d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:03.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34d57fa640 1 -- 172.21.15.52:0/1671453946 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0014620 con 0x7f34d815ea70 2024-05-04T12:19:03.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34d57fa640 1 -- 172.21.15.52:0/1671453946 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d0019070 con 0x7f34d815ea70 2024-05-04T12:19:03.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.441+0000 7f34d57fa640 1 -- 172.21.15.52:0/1671453946 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0014a20 con 0x7f34d815ea70 2024-05-04T12:19:03.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.445+0000 7f34dea9c640 1 -- 172.21.15.52:0/1671453946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d80af220 con 0x7f34d815ea70 2024-05-04T12:19:03.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.445+0000 7f34dea9c640 1 -- 172.21.15.52:0/1671453946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d80af7e0 con 0x7f34d815ea70 2024-05-04T12:19:03.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.445+0000 7f34dea9c640 1 -- 172.21.15.52:0/1671453946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d8062e50 con 0x7f34d815ea70 2024-05-04T12:19:03.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.445+0000 7f34d57fa640 1 -- 172.21.15.52:0/1671453946 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f34d0014bc0 con 0x7f34d815ea70 2024-05-04T12:19:03.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.445+0000 7f34d57fa640 1 --2- 172.21.15.52:0/1671453946 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f34c40678c0 0x7f34c4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.445+0000 7f34d57fa640 1 -- 172.21.15.52:0/1671453946 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f34d00866e0 con 0x7f34d815ea70 2024-05-04T12:19:03.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.445+0000 7f34dc838640 1 --2- 172.21.15.52:0/1671453946 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f34c40678c0 0x7f34c4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.445+0000 7f34dc838640 1 --2- 172.21.15.52:0/1671453946 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f34c40678c0 0x7f34c4069d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f34c8007fd0 tx=0x7f34c8007980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:03.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.449+0000 7f34d57fa640 1 -- 172.21.15.52:0/1671453946 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f34d0011260 con 0x7f34d815ea70 2024-05-04T12:19:03.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.609+0000 7f34dea9c640 1 -- 172.21.15.52:0/1671453946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f34d8075320 con 0x7f34d815ea70 2024-05-04T12:19:03.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.609+0000 7f34d57fa640 1 -- 172.21.15.52:0/1671453946 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f34d00570f0 con 0x7f34d815ea70 2024-05-04T12:19:03.613 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:19:03.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.613+0000 7f34dea9c640 1 -- 172.21.15.52:0/1671453946 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f34c40678c0 msgr2=0x7f34c4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.613+0000 7f34dea9c640 1 --2- 172.21.15.52:0/1671453946 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f34c40678c0 0x7f34c4069d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f34c8007fd0 tx=0x7f34c8007980 comp rx=0 tx=0).stop 2024-05-04T12:19:03.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.613+0000 7f34dea9c640 1 -- 172.21.15.52:0/1671453946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34d815ea70 msgr2=0x7f34d8140580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.613+0000 7f34dea9c640 1 --2- 172.21.15.52:0/1671453946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34d815ea70 0x7f34d8140580 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f34d0007e70 tx=0x7f34d000e4b0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.613+0000 7f34dea9c640 1 -- 172.21.15.52:0/1671453946 shutdown_connections 2024-05-04T12:19:03.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.613+0000 7f34dea9c640 1 --2- 172.21.15.52:0/1671453946 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f34c40678c0 0x7f34c4069d80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.613+0000 7f34dea9c640 1 --2- 172.21.15.52:0/1671453946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34d815ea70 0x7f34d8140580 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.613+0000 7f34dea9c640 1 --2- 172.21.15.52:0/1671453946 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f34d8147280 0x7f34d8140040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.613+0000 7f34dea9c640 1 --2- 172.21.15.52:0/1671453946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d806cdd0 0x7f34d80aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.613+0000 7f34dea9c640 1 -- 172.21.15.52:0/1671453946 >> 172.21.15.52:0/1671453946 conn(0x7f34d806b210 msgr2=0x7f34d8146530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:03.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.613+0000 7f34dea9c640 1 -- 172.21.15.52:0/1671453946 shutdown_connections 2024-05-04T12:19:03.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.613+0000 7f34dea9c640 1 -- 172.21.15.52:0/1671453946 wait complete. 2024-05-04T12:19:03.634 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-04T12:19:03.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.757+0000 7f27f292b640 1 Processor -- start 2024-05-04T12:19:03.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.757+0000 7f27f292b640 1 -- start start 2024-05-04T12:19:03.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f292b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f27ec1608d0 0x7f27ec15cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f292b640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27ec06dd90 con 0x7f27ec070d10 2024-05-04T12:19:03.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f292b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27ec06e000 con 0x7f27ec070930 2024-05-04T12:19:03.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f292b640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f27ec06e2e0 con 0x7f27ec1608d0 2024-05-04T12:19:03.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f0ec8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f27ec1608d0 0x7f27ec15cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27e37fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f27ec070d10 0x7f27ec15c870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:49998/0 (socket says 172.21.15.52:49998) 2024-05-04T12:19:03.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27e37fe640 1 -- 172.21.15.52:0/2828074442 learned_addr learned my addr 172.21.15.52:0/2828074442 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:03.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f0ec8640 1 --2- 172.21.15.52:0/2828074442 >> v2:172.21.15.52:3300/0 conn(0x7f27ec1608d0 0x7f27ec15cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37834/0 (socket says 172.21.15.52:37834) 2024-05-04T12:19:03.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f0ec8640 1 -- 172.21.15.52:0/2828074442 >> v1:172.21.15.111:6789/0 conn(0x7f27ec070930 legacy=0x7f27ec1601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f0ec8640 1 -- 172.21.15.52:0/2828074442 >> v1:172.21.15.52:6789/0 conn(0x7f27ec070d10 legacy=0x7f27ec15c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f0ec8640 1 -- 172.21.15.52:0/2828074442 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec06e5c0 con 0x7f27ec1608d0 2024-05-04T12:19:03.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f0ec8640 1 --2- 172.21.15.52:0/2828074442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27ec1608d0 0x7f27ec15cf80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009d90 tx=0x7f27e401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806e1f85f9f8bc5e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:03.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27eb7fe640 1 -- 172.21.15.52:0/2828074442 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e40b2070 con 0x7f27ec1608d0 2024-05-04T12:19:03.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27eb7fe640 1 -- 172.21.15.52:0/2828074442 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e40a58c0 con 0x7f27ec1608d0 2024-05-04T12:19:03.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27eb7fe640 1 -- 172.21.15.52:0/2828074442 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e40a5c90 con 0x7f27ec1608d0 2024-05-04T12:19:03.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f292b640 1 -- 172.21.15.52:0/2828074442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27ec1608d0 msgr2=0x7f27ec15cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f292b640 1 --2- 172.21.15.52:0/2828074442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27ec1608d0 0x7f27ec15cf80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009d90 tx=0x7f27e401c8e0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f292b640 1 -- 172.21.15.52:0/2828074442 shutdown_connections 2024-05-04T12:19:03.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f292b640 1 --2- 172.21.15.52:0/2828074442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27ec1608d0 0x7f27ec15cf80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f292b640 1 -- 172.21.15.52:0/2828074442 >> 172.21.15.52:0/2828074442 conn(0x7f27ec067070 msgr2=0x7f27ec067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:03.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f292b640 1 -- 172.21.15.52:0/2828074442 shutdown_connections 2024-05-04T12:19:03.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.761+0000 7f27f292b640 1 -- 172.21.15.52:0/2828074442 wait complete. 2024-05-04T12:19:03.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f292b640 1 Processor -- start 2024-05-04T12:19:03.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f292b640 1 -- start start 2024-05-04T12:19:03.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f292b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f27ec070930 0x7f27ec13f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f292b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27ec070d10 0x7f27ec154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f292b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27ec1608d0 0x7f27ec13de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f292b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec169c40 con 0x7f27ec1608d0 2024-05-04T12:19:03.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f292b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec169ac0 con 0x7f27ec070d10 2024-05-04T12:19:03.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f292b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f27ec169dc0 con 0x7f27ec070930 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f0ec8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27ec1608d0 0x7f27ec13de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f0ec8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27ec1608d0 0x7f27ec13de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37846/0 (socket says 172.21.15.52:37846) 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f0ec8640 1 -- 172.21.15.52:0/1298946237 learned_addr learned my addr 172.21.15.52:0/1298946237 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27ebfff640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f27ec070930 0x7f27ec13f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27e37fe640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27ec070d10 0x7f27ec154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f0ec8640 1 -- 172.21.15.52:0/1298946237 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f27ec070930 msgr2=0x7f27ec13f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f0ec8640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f27ec070930 0x7f27ec13f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f0ec8640 1 -- 172.21.15.52:0/1298946237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27ec070d10 msgr2=0x7f27ec154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f0ec8640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27ec070d10 0x7f27ec154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f0ec8640 1 -- 172.21.15.52:0/1298946237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec13cea0 con 0x7f27ec1608d0 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27e37fe640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27ec070d10 0x7f27ec154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f0ec8640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27ec1608d0 0x7f27ec13de80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f27e401a510 tx=0x7f27e4019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27e97fa640 1 -- 172.21.15.52:0/1298946237 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e40b2050 con 0x7f27ec1608d0 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27e97fa640 1 -- 172.21.15.52:0/1298946237 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e40be070 con 0x7f27ec1608d0 2024-05-04T12:19:03.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f292b640 1 -- 172.21.15.52:0/1298946237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ec13d180 con 0x7f27ec1608d0 2024-05-04T12:19:03.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27e97fa640 1 -- 172.21.15.52:0/1298946237 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e40ada50 con 0x7f27ec1608d0 2024-05-04T12:19:03.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f292b640 1 -- 172.21.15.52:0/1298946237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ec135450 con 0x7f27ec1608d0 2024-05-04T12:19:03.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.765+0000 7f27f292b640 1 -- 172.21.15.52:0/1298946237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27ec13d6a0 con 0x7f27ec1608d0 2024-05-04T12:19:03.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.769+0000 7f27e97fa640 1 -- 172.21.15.52:0/1298946237 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f27e40adbf0 con 0x7f27ec1608d0 2024-05-04T12:19:03.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.769+0000 7f27e97fa640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f27cc0678c0 0x7f27cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:03.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.769+0000 7f27e97fa640 1 -- 172.21.15.52:0/1298946237 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f27e4120fe0 con 0x7f27ec1608d0 2024-05-04T12:19:03.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.769+0000 7f27ebfff640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f27cc0678c0 0x7f27cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:03.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.769+0000 7f27ebfff640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f27cc0678c0 0x7f27cc069d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f27dc000f70 tx=0x7f27dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:03.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.773+0000 7f27e97fa640 1 -- 172.21.15.52:0/1298946237 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f27ec13d6a0 con 0x7f27ec1608d0 2024-05-04T12:19:03.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.945+0000 7f27f292b640 1 -- 172.21.15.52:0/1298946237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f27ec135650 con 0x7f27ec1608d0 2024-05-04T12:19:03.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.949+0000 7f27e97fa640 1 -- 172.21.15.52:0/1298946237 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f27ec135650 con 0x7f27ec1608d0 2024-05-04T12:19:03.951 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-04T12:19:03.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.949+0000 7f27f292b640 1 -- 172.21.15.52:0/1298946237 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f27cc0678c0 msgr2=0x7f27cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.949+0000 7f27f292b640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f27cc0678c0 0x7f27cc069d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f27dc000f70 tx=0x7f27dc009040 comp rx=0 tx=0).stop 2024-05-04T12:19:03.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.949+0000 7f27f292b640 1 -- 172.21.15.52:0/1298946237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27ec1608d0 msgr2=0x7f27ec13de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:03.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.949+0000 7f27f292b640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27ec1608d0 0x7f27ec13de80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f27e401a510 tx=0x7f27e4019680 comp rx=0 tx=0).stop 2024-05-04T12:19:03.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.949+0000 7f27f292b640 1 -- 172.21.15.52:0/1298946237 shutdown_connections 2024-05-04T12:19:03.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.949+0000 7f27f292b640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f27cc0678c0 0x7f27cc069d80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.949+0000 7f27f292b640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27ec1608d0 0x7f27ec13de80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.949+0000 7f27f292b640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27ec070d10 0x7f27ec154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.949+0000 7f27f292b640 1 --2- 172.21.15.52:0/1298946237 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f27ec070930 0x7f27ec13f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:03.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.949+0000 7f27f292b640 1 -- 172.21.15.52:0/1298946237 >> 172.21.15.52:0/1298946237 conn(0x7f27ec067070 msgr2=0x7f27ec0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:03.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.949+0000 7f27f292b640 1 -- 172.21.15.52:0/1298946237 shutdown_connections 2024-05-04T12:19:03.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:03.949+0000 7f27f292b640 1 -- 172.21.15.52:0/1298946237 wait complete. 2024-05-04T12:19:03.969 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:19:03.969 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T12:19:03.975 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:19:04.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00d0fe7640 1 Processor -- start 2024-05-04T12:19:04.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00d0fe7640 1 -- start start 2024-05-04T12:19:04.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00d0fe7640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00cc0722f0 0x7f00cc15e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00d0fe7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f00cc15a910 0x7f00cc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00d0fe7640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00cc15ea30 con 0x7f00cc15a530 2024-05-04T12:19:04.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00d0fe7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f00cc06f080 con 0x7f00cc15a910 2024-05-04T12:19:04.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00d0fe7640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f00cc15ed10 con 0x7f00cc0722f0 2024-05-04T12:19:04.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00ca59c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f00cc15a530 0x7f00cc071be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:50010/0 (socket says 172.21.15.52:50010) 2024-05-04T12:19:04.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00ca59c640 1 -- 172.21.15.52:0/2269652432 learned_addr learned my addr 172.21.15.52:0/2269652432 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:04.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00cad9d640 1 --2- 172.21.15.52:0/2269652432 >> v2:172.21.15.111:3300/0 conn(0x7f00cc15a910 0x7f00cc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c9d9b640 1 --2- 172.21.15.52:0/2269652432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00cc0722f0 0x7f00cc15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 -- 172.21.15.52:0/2269652432 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2856670343 0 0) 0x7f00cc15ea30 con 0x7f00cc15a530 2024-05-04T12:19:04.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 -- 172.21.15.52:0/2269652432 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00b8004640 con 0x7f00cc15a530 2024-05-04T12:19:04.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 -- 172.21.15.52:0/2269652432 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1020951147 0 0) 0x7f00b8004640 con 0x7f00cc15a530 2024-05-04T12:19:04.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 -- 172.21.15.52:0/2269652432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00cc0722f0 msgr2=0x7f00cc15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:04.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 --2- 172.21.15.52:0/2269652432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00cc0722f0 0x7f00cc15e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 -- 172.21.15.52:0/2269652432 >> v2:172.21.15.111:3300/0 conn(0x7f00cc15a910 msgr2=0x7f00cc06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:04.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 --2- 172.21.15.52:0/2269652432 >> v2:172.21.15.111:3300/0 conn(0x7f00cc15a910 0x7f00cc06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 -- 172.21.15.52:0/2269652432 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc15eff0 con 0x7f00cc15a530 2024-05-04T12:19:04.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 -- 172.21.15.52:0/2269652432 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f00bc001680 con 0x7f00cc15a530 2024-05-04T12:19:04.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 -- 172.21.15.52:0/2269652432 >> v1:172.21.15.52:6789/0 conn(0x7f00cc15a530 legacy=0x7f00cc071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 --2- 172.21.15.52:0/2269652432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00b8004ad0 0x7f00b8004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 --2- 172.21.15.52:0/2269652432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00b8005920 0x7f00b8005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 --2- 172.21.15.52:0/2269652432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00cc0722f0 0x7f00b80062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 -- 172.21.15.52:0/2269652432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc06f080 con 0x7f00b8004ad0 2024-05-04T12:19:04.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 -- 172.21.15.52:0/2269652432 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc15ed10 con 0x7f00b8005920 2024-05-04T12:19:04.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c959a640 1 -- 172.21.15.52:0/2269652432 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f00b8003450 con 0x7f00cc0722f0 2024-05-04T12:19:04.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00ca59c640 1 --2- 172.21.15.52:0/2269652432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00b8004ad0 0x7f00b8004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00cad9d640 1 --2- 172.21.15.52:0/2269652432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00cc0722f0 0x7f00b80062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00c9d9b640 1 --2- 172.21.15.52:0/2269652432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00b8005920 0x7f00b8005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00ca59c640 1 -- 172.21.15.52:0/2269652432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00cc0722f0 msgr2=0x7f00b80062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:04.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00ca59c640 1 --2- 172.21.15.52:0/2269652432 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00cc0722f0 0x7f00b80062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00ca59c640 1 -- 172.21.15.52:0/2269652432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00b8005920 msgr2=0x7f00b8005da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:04.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00ca59c640 1 --2- 172.21.15.52:0/2269652432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00b8005920 0x7f00b8005da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.137+0000 7f00ca59c640 1 -- 172.21.15.52:0/2269652432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b8004640 con 0x7f00b8004ad0 2024-05-04T12:19:04.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00ca59c640 1 --2- 172.21.15.52:0/2269652432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00b8004ad0 0x7f00b8004ef0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00c210 tx=0x7f00bc01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2419338954e4276 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:04.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00c959a640 1 -- 172.21.15.52:0/2269652432 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc007d60 con 0x7f00b8004ad0 2024-05-04T12:19:04.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00c959a640 1 -- 172.21.15.52:0/2269652432 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00bc0a9020 con 0x7f00b8004ad0 2024-05-04T12:19:04.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00c959a640 1 -- 172.21.15.52:0/2269652432 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc005b20 con 0x7f00b8004ad0 2024-05-04T12:19:04.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00d0fe7640 1 -- 172.21.15.52:0/2269652432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00b8004ad0 msgr2=0x7f00b8004ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00d0fe7640 1 --2- 172.21.15.52:0/2269652432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00b8004ad0 0x7f00b8004ef0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00c210 tx=0x7f00bc01cca0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00cad9d640 1 -- 172.21.15.52:0/2269652432 reap_dead start 2024-05-04T12:19:04.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00d0fe7640 1 -- 172.21.15.52:0/2269652432 shutdown_connections 2024-05-04T12:19:04.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00d0fe7640 1 -- 172.21.15.52:0/2269652432 >> 172.21.15.52:0/2269652432 conn(0x7f00cc0b14a0 msgr2=0x7f00cc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:04.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00d0fe7640 1 -- 172.21.15.52:0/2269652432 shutdown_connections 2024-05-04T12:19:04.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00d0fe7640 1 -- 172.21.15.52:0/2269652432 wait complete. 2024-05-04T12:19:04.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00d0fe7640 1 Processor -- start 2024-05-04T12:19:04.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00d0fe7640 1 -- start start 2024-05-04T12:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00d0fe7640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00b8004ad0 0x7f00cc1f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00d0fe7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00cc155930 0x7f00cc155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00d0fe7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00cc0b04d0 0x7f00cc0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00d0fe7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc06dcf0 con 0x7f00cc0b04d0 2024-05-04T12:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00d0fe7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc06db70 con 0x7f00cc155930 2024-05-04T12:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00d0fe7640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f00cc06de70 con 0x7f00b8004ad0 2024-05-04T12:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00cad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00cc0b04d0 0x7f00cc0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00cad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00cc0b04d0 0x7f00cc0b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37864/0 (socket says 172.21.15.52:37864) 2024-05-04T12:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00cad9d640 1 -- 172.21.15.52:0/1231852963 learned_addr learned my addr 172.21.15.52:0/1231852963 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00c9d9b640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00cc155930 0x7f00cc155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.141+0000 7f00ca59c640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00b8004ad0 0x7f00cc1f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00cad9d640 1 -- 172.21.15.52:0/1231852963 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00b8004ad0 msgr2=0x7f00cc1f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00cad9d640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00b8004ad0 0x7f00cc1f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00cad9d640 1 -- 172.21.15.52:0/1231852963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00cc155930 msgr2=0x7f00cc155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00cad9d640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00cc155930 0x7f00cc155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00cad9d640 1 -- 172.21.15.52:0/1231852963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc14efd0 con 0x7f00cc0b04d0 2024-05-04T12:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00ca59c640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00b8004ad0 0x7f00cc1f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00cad9d640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00cc0b04d0 0x7f00cc0b0950 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f00cc06cf00 tx=0x7f00c40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00c2ffd640 1 -- 172.21.15.52:0/1231852963 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4011e10 con 0x7f00cc0b04d0 2024-05-04T12:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00c2ffd640 1 -- 172.21.15.52:0/1231852963 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c401b070 con 0x7f00cc0b04d0 2024-05-04T12:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00d0fe7640 1 -- 172.21.15.52:0/1231852963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00cc1f7710 con 0x7f00cc0b04d0 2024-05-04T12:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00c2ffd640 1 -- 172.21.15.52:0/1231852963 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c40039a0 con 0x7f00cc0b04d0 2024-05-04T12:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00d0fe7640 1 -- 172.21.15.52:0/1231852963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00cc1f7cd0 con 0x7f00cc0b04d0 2024-05-04T12:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00d0fe7640 1 -- 172.21.15.52:0/1231852963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00cc1f7490 con 0x7f00cc0b04d0 2024-05-04T12:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00c2ffd640 1 -- 172.21.15.52:0/1231852963 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f00c4003b60 con 0x7f00cc0b04d0 2024-05-04T12:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00c2ffd640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f00ac0678c0 0x7f00ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00c2ffd640 1 -- 172.21.15.52:0/1231852963 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f00c40863e0 con 0x7f00cc0b04d0 2024-05-04T12:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00ca59c640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f00ac0678c0 0x7f00ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.145+0000 7f00ca59c640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f00ac0678c0 0x7f00ac069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0aa000 tx=0x7f00bc000ff0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:04.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.149+0000 7f00c2ffd640 1 -- 172.21.15.52:0/1231852963 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f00cc1f7490 con 0x7f00cc0b04d0 2024-05-04T12:19:04.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.305+0000 7f00d0fe7640 1 -- 172.21.15.52:0/1231852963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00cc062e50 con 0x7f00cc0b04d0 2024-05-04T12:19:04.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.309+0000 7f00c2ffd640 1 -- 172.21.15.52:0/1231852963 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16913 (secure 0 0 0) 0x7f00cc14efd0 con 0x7f00cc0b04d0 2024-05-04T12:19:04.312 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:04.312 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:19:04.046290+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:19:04.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.309+0000 7f00d0fe7640 1 -- 172.21.15.52:0/1231852963 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f00ac0678c0 msgr2=0x7f00ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.309+0000 7f00d0fe7640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f00ac0678c0 0x7f00ac069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0aa000 tx=0x7f00bc000ff0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.309+0000 7f00d0fe7640 1 -- 172.21.15.52:0/1231852963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00cc0b04d0 msgr2=0x7f00cc0b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.309+0000 7f00d0fe7640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00cc0b04d0 0x7f00cc0b0950 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f00cc06cf00 tx=0x7f00c40115a0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.309+0000 7f00d0fe7640 1 -- 172.21.15.52:0/1231852963 shutdown_connections 2024-05-04T12:19:04.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.309+0000 7f00d0fe7640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f00ac0678c0 0x7f00ac069d80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.309+0000 7f00d0fe7640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00cc0b04d0 0x7f00cc0b0950 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.309+0000 7f00d0fe7640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00cc155930 0x7f00cc155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.309+0000 7f00d0fe7640 1 --2- 172.21.15.52:0/1231852963 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00b8004ad0 0x7f00cc1f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.309+0000 7f00d0fe7640 1 -- 172.21.15.52:0/1231852963 >> 172.21.15.52:0/1231852963 conn(0x7f00cc0b14a0 msgr2=0x7f00cc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:04.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.309+0000 7f00d0fe7640 1 -- 172.21.15.52:0/1231852963 shutdown_connections 2024-05-04T12:19:04.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.309+0000 7f00d0fe7640 1 -- 172.21.15.52:0/1231852963 wait complete. 2024-05-04T12:19:04.329 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:19:04.330 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:19:04.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa360f78640 1 Processor -- start 2024-05-04T12:19:04.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa360f78640 1 -- start start 2024-05-04T12:19:04.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa360f78640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa35c141010 0x7fa35c0d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa360f78640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa35c0d2540 0x7fa35c1418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa360f78640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa35c0d29c0 con 0x7fa35c144570 2024-05-04T12:19:04.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa360f78640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fa35c0d2ca0 con 0x7fa35c0d2540 2024-05-04T12:19:04.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa360f78640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa35c0d2f80 con 0x7fa35c141010 2024-05-04T12:19:04.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa35a59c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa35c141010 0x7fa35c0d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa35ad9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa35c0d2540 0x7fa35c1418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa359d9b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa35c144570 0x7fa35c143940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.52:38218/0 (socket says 172.21.15.52:38218) 2024-05-04T12:19:04.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa359d9b640 1 -- 172.21.15.52:0/2614487684 learned_addr learned my addr 172.21.15.52:0/2614487684 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:04.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa35ad9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa35c0d2540 0x7fa35c1418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:60886/0 (socket says 172.21.15.52:60886) 2024-05-04T12:19:04.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa35a59c640 1 -- 172.21.15.52:0/2614487684 >> v2:172.21.15.111:3300/0 conn(0x7fa35c0d2540 msgr2=0x7fa35c1418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:04.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa35a59c640 1 --2- 172.21.15.52:0/2614487684 >> v2:172.21.15.111:3300/0 conn(0x7fa35c0d2540 0x7fa35c1418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa35a59c640 1 -- 172.21.15.52:0/2614487684 >> v1:172.21.15.111:6789/0 conn(0x7fa35c144570 legacy=0x7fa35c143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa35a59c640 1 -- 172.21.15.52:0/2614487684 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c0d3260 con 0x7fa35c141010 2024-05-04T12:19:04.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.445+0000 7fa35ad9d640 1 --2- 172.21.15.52:0/2614487684 >> v2:172.21.15.111:3300/0 conn(0x7fa35c0d2540 0x7fa35c1418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:19:04.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.449+0000 7fa35a59c640 1 --2- 172.21.15.52:0/2614487684 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa35c141010 0x7fa35c0d2000 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0097c0 tx=0x7fa34c01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=855ce80f179d15ad server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:04.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.449+0000 7fa35959a640 1 -- 172.21.15.52:0/2614487684 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0ab600 con 0x7fa35c141010 2024-05-04T12:19:04.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.449+0000 7fa35959a640 1 -- 172.21.15.52:0/2614487684 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34c0b0070 con 0x7fa35c141010 2024-05-04T12:19:04.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.449+0000 7fa35959a640 1 -- 172.21.15.52:0/2614487684 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0ab9c0 con 0x7fa35c141010 2024-05-04T12:19:04.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.449+0000 7fa360f78640 1 -- 172.21.15.52:0/2614487684 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa35c141010 msgr2=0x7fa35c0d2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.449+0000 7fa360f78640 1 --2- 172.21.15.52:0/2614487684 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa35c141010 0x7fa35c0d2000 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0097c0 tx=0x7fa34c01c3e0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.449+0000 7fa360f78640 1 -- 172.21.15.52:0/2614487684 shutdown_connections 2024-05-04T12:19:04.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.449+0000 7fa360f78640 1 --2- 172.21.15.52:0/2614487684 >> v2:172.21.15.111:3300/0 conn(0x7fa35c0d2540 0x7fa35c1418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.449+0000 7fa360f78640 1 --2- 172.21.15.52:0/2614487684 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa35c141010 0x7fa35c0d2000 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.449+0000 7fa360f78640 1 -- 172.21.15.52:0/2614487684 >> 172.21.15.52:0/2614487684 conn(0x7fa35c072be0 msgr2=0x7fa35c0d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:04.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.449+0000 7fa360f78640 1 -- 172.21.15.52:0/2614487684 shutdown_connections 2024-05-04T12:19:04.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.449+0000 7fa360f78640 1 -- 172.21.15.52:0/2614487684 wait complete. 2024-05-04T12:19:04.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.449+0000 7fa360f78640 1 Processor -- start 2024-05-04T12:19:04.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.449+0000 7fa360f78640 1 -- start start 2024-05-04T12:19:04.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa360f78640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa35c0d2540 0x7fa35c157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa360f78640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa35c141010 0x7fa35c13e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa360f78640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa35c144570 0x7fa35c157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa360f78640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c142600 con 0x7fa35c141010 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa360f78640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c144260 con 0x7fa35c0d2540 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa360f78640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa35c142780 con 0x7fa35c144570 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa359d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa35c141010 0x7fa35c13e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa359d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa35c141010 0x7fa35c13e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37880/0 (socket says 172.21.15.52:37880) 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa359d9b640 1 -- 172.21.15.52:0/2717193254 learned_addr learned my addr 172.21.15.52:0/2717193254 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa35a59c640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa35c0d2540 0x7fa35c157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa35ad9d640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa35c144570 0x7fa35c157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa359d9b640 1 -- 172.21.15.52:0/2717193254 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa35c144570 msgr2=0x7fa35c157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa359d9b640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa35c144570 0x7fa35c157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa359d9b640 1 -- 172.21.15.52:0/2717193254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa35c0d2540 msgr2=0x7fa35c157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa359d9b640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa35c0d2540 0x7fa35c157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa359d9b640 1 -- 172.21.15.52:0/2717193254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c070600 con 0x7fa35c141010 2024-05-04T12:19:04.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa35ad9d640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa35c144570 0x7fa35c157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:04.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa359d9b640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa35c141010 0x7fa35c13e630 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa35000c370 tx=0x7fa35000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:04.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa34affd640 1 -- 172.21.15.52:0/2717193254 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350017070 con 0x7fa35c141010 2024-05-04T12:19:04.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa360f78640 1 -- 172.21.15.52:0/2717193254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c157760 con 0x7fa35c141010 2024-05-04T12:19:04.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa34affd640 1 -- 172.21.15.52:0/2717193254 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa350005cf0 con 0x7fa35c141010 2024-05-04T12:19:04.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa34affd640 1 -- 172.21.15.52:0/2717193254 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3500072f0 con 0x7fa35c141010 2024-05-04T12:19:04.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa360f78640 1 -- 172.21.15.52:0/2717193254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c13fd30 con 0x7fa35c141010 2024-05-04T12:19:04.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa360f78640 1 -- 172.21.15.52:0/2717193254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35c062e50 con 0x7fa35c141010 2024-05-04T12:19:04.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa34affd640 1 -- 172.21.15.52:0/2717193254 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa35001a070 con 0x7fa35c141010 2024-05-04T12:19:04.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa34affd640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa33c0678c0 0x7fa33c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa34affd640 1 -- 172.21.15.52:0/2717193254 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7fa350085140 con 0x7fa35c141010 2024-05-04T12:19:04.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.453+0000 7fa35a59c640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa33c0678c0 0x7fa33c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.457+0000 7fa35a59c640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa33c0678c0 0x7fa33c069d80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0af000 tx=0x7fa34c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:04.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.457+0000 7fa34affd640 1 -- 172.21.15.52:0/2717193254 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa350055ae0 con 0x7fa35c141010 2024-05-04T12:19:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.613+0000 7fa360f78640 1 -- 172.21.15.52:0/2717193254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa35c0707d0 con 0x7fa35c141010 2024-05-04T12:19:04.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.613+0000 7fa34affd640 1 -- 172.21.15.52:0/2717193254 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fa3500553e0 con 0x7fa35c141010 2024-05-04T12:19:04.618 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:19:04.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.617+0000 7fa360f78640 1 -- 172.21.15.52:0/2717193254 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa33c0678c0 msgr2=0x7fa33c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.617+0000 7fa360f78640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa33c0678c0 0x7fa33c069d80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0af000 tx=0x7fa34c0a7000 comp rx=0 tx=0).stop 2024-05-04T12:19:04.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.617+0000 7fa360f78640 1 -- 172.21.15.52:0/2717193254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa35c141010 msgr2=0x7fa35c13e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.617+0000 7fa360f78640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa35c141010 0x7fa35c13e630 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa35000c370 tx=0x7fa35000c830 comp rx=0 tx=0).stop 2024-05-04T12:19:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.617+0000 7fa360f78640 1 -- 172.21.15.52:0/2717193254 shutdown_connections 2024-05-04T12:19:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.617+0000 7fa360f78640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa33c0678c0 0x7fa33c069d80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.617+0000 7fa360f78640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa35c144570 0x7fa35c157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.617+0000 7fa360f78640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa35c141010 0x7fa35c13e630 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.617+0000 7fa360f78640 1 --2- 172.21.15.52:0/2717193254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa35c0d2540 0x7fa35c157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.617+0000 7fa360f78640 1 -- 172.21.15.52:0/2717193254 >> 172.21.15.52:0/2717193254 conn(0x7fa35c072be0 msgr2=0x7fa35c0d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.617+0000 7fa360f78640 1 -- 172.21.15.52:0/2717193254 shutdown_connections 2024-05-04T12:19:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.617+0000 7fa360f78640 1 -- 172.21.15.52:0/2717193254 wait complete. 2024-05-04T12:19:04.636 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:19:04.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd609466640 1 Processor -- start 2024-05-04T12:19:04.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd609466640 1 -- start start 2024-05-04T12:19:04.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd609466640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd60415bd90 0x7fd6040cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd609466640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd60406fa80 0x7fd6040cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd609466640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6040ccb80 con 0x7fd60415c170 2024-05-04T12:19:04.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd609466640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fd6040cc930 con 0x7fd60415bd90 2024-05-04T12:19:04.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd609466640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fd6040cd4b0 con 0x7fd60406fa80 2024-05-04T12:19:04.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd602ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd60415bd90 0x7fd6040cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd602ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd60415bd90 0x7fd6040cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37884/0 (socket says 172.21.15.52:37884) 2024-05-04T12:19:04.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd602ffd640 1 -- 172.21.15.52:0/2948161349 learned_addr learned my addr 172.21.15.52:0/2948161349 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:04.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd602ffd640 1 -- 172.21.15.52:0/2948161349 >> v2:172.21.15.111:3300/0 conn(0x7fd60406fa80 msgr2=0x7fd6040cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:04.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd6037fe640 1 --2- 172.21.15.52:0/2948161349 >> v2:172.21.15.111:3300/0 conn(0x7fd60406fa80 0x7fd6040cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd602ffd640 1 --2- 172.21.15.52:0/2948161349 >> v2:172.21.15.111:3300/0 conn(0x7fd60406fa80 0x7fd6040cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd602ffd640 1 -- 172.21.15.52:0/2948161349 >> v1:172.21.15.111:6789/0 conn(0x7fd60415c170 legacy=0x7fd60406f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd602ffd640 1 -- 172.21.15.52:0/2948161349 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6040cce60 con 0x7fd60415bd90 2024-05-04T12:19:04.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.741+0000 7fd6037fe640 1 --2- 172.21.15.52:0/2948161349 >> v2:172.21.15.111:3300/0 conn(0x7fd60406fa80 0x7fd6040cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:04.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.745+0000 7fd602ffd640 1 --2- 172.21.15.52:0/2948161349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd60415bd90 0x7fd6040cf060 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd6040aedc0 tx=0x7fd5f401eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55264115305e2ea8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:04.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.745+0000 7fd6027fc640 1 -- 172.21.15.52:0/2948161349 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40b4070 con 0x7fd60415bd90 2024-05-04T12:19:04.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.745+0000 7fd6027fc640 1 -- 172.21.15.52:0/2948161349 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f40a7a40 con 0x7fd60415bd90 2024-05-04T12:19:04.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.745+0000 7fd6027fc640 1 -- 172.21.15.52:0/2948161349 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40a7e10 con 0x7fd60415bd90 2024-05-04T12:19:04.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.745+0000 7fd609466640 1 -- 172.21.15.52:0/2948161349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd60415bd90 msgr2=0x7fd6040cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.745+0000 7fd609466640 1 --2- 172.21.15.52:0/2948161349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd60415bd90 0x7fd6040cf060 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd6040aedc0 tx=0x7fd5f401eb60 comp rx=0 tx=0).stop 2024-05-04T12:19:04.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.745+0000 7fd609466640 1 -- 172.21.15.52:0/2948161349 shutdown_connections 2024-05-04T12:19:04.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.745+0000 7fd609466640 1 --2- 172.21.15.52:0/2948161349 >> v2:172.21.15.111:3300/0 conn(0x7fd60406fa80 0x7fd6040cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.745+0000 7fd609466640 1 --2- 172.21.15.52:0/2948161349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd60415bd90 0x7fd6040cf060 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.745+0000 7fd609466640 1 -- 172.21.15.52:0/2948161349 >> 172.21.15.52:0/2948161349 conn(0x7fd604067730 msgr2=0x7fd6040b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:04.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.745+0000 7fd609466640 1 -- 172.21.15.52:0/2948161349 shutdown_connections 2024-05-04T12:19:04.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.745+0000 7fd609466640 1 -- 172.21.15.52:0/2948161349 wait complete. 2024-05-04T12:19:04.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.745+0000 7fd609466640 1 Processor -- start 2024-05-04T12:19:04.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.745+0000 7fd609466640 1 -- start start 2024-05-04T12:19:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd609466640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd60406fa80 0x7fd604152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd609466640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd60415bd90 0x7fd604145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd609466640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd60415c170 0x7fd60415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd609466640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd6040ceb70 con 0x7fd60406fa80 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd609466640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd6040ce9f0 con 0x7fd60415c170 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd609466640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fd6040cecf0 con 0x7fd60415bd90 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd602ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd60406fa80 0x7fd604152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd602ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd60406fa80 0x7fd604152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37898/0 (socket says 172.21.15.52:37898) 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd602ffd640 1 -- 172.21.15.52:0/4185974390 learned_addr learned my addr 172.21.15.52:0/4185974390 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd6037fe640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd60415c170 0x7fd60415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd602ffd640 1 -- 172.21.15.52:0/4185974390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd60415bd90 msgr2=0x7fd604145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd5fa7fc640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd60415bd90 0x7fd604145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd602ffd640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd60415bd90 0x7fd604145750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd602ffd640 1 -- 172.21.15.52:0/4185974390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd60415c170 msgr2=0x7fd60415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd602ffd640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd60415c170 0x7fd60415cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd602ffd640 1 -- 172.21.15.52:0/4185974390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6041498e0 con 0x7fd60406fa80 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd6037fe640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd60415c170 0x7fd60415cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd602ffd640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd60406fa80 0x7fd604152fd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400b2c0 tx=0x7fd5f401b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:04.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd5fbfff640 1 -- 172.21.15.52:0/4185974390 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40be070 con 0x7fd60406fa80 2024-05-04T12:19:04.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd5fbfff640 1 -- 172.21.15.52:0/4185974390 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f40afc80 con 0x7fd60406fa80 2024-05-04T12:19:04.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd5fbfff640 1 -- 172.21.15.52:0/4185974390 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40b4040 con 0x7fd60406fa80 2024-05-04T12:19:04.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd609466640 1 -- 172.21.15.52:0/4185974390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd604149bc0 con 0x7fd60406fa80 2024-05-04T12:19:04.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd609466640 1 -- 172.21.15.52:0/4185974390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60414a180 con 0x7fd60406fa80 2024-05-04T12:19:04.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd609466640 1 -- 172.21.15.52:0/4185974390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd604062e50 con 0x7fd60406fa80 2024-05-04T12:19:04.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd5fbfff640 1 -- 172.21.15.52:0/4185974390 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd5f40c1070 con 0x7fd60406fa80 2024-05-04T12:19:04.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd5fbfff640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd5e40678c0 0x7fd5e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:04.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd5fbfff640 1 -- 172.21.15.52:0/4185974390 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7fd5f4123010 con 0x7fd60406fa80 2024-05-04T12:19:04.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd5fa7fc640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd5e40678c0 0x7fd5e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:04.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.749+0000 7fd5fa7fc640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd5e40678c0 0x7fd5e4069d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd60415d7a0 tx=0x7fd5ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:04.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.753+0000 7fd5fbfff640 1 -- 172.21.15.52:0/4185974390 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd5f40187e0 con 0x7fd60406fa80 2024-05-04T12:19:04.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.913+0000 7fd609466640 1 -- 172.21.15.52:0/4185974390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd604075320 con 0x7fd60406fa80 2024-05-04T12:19:04.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.917+0000 7fd5fbfff640 1 -- 172.21.15.52:0/4185974390 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fd5f40f3ac0 con 0x7fd60406fa80 2024-05-04T12:19:04.919 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:19:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.917+0000 7fd609466640 1 -- 172.21.15.52:0/4185974390 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd5e40678c0 msgr2=0x7fd5e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.917+0000 7fd609466640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd5e40678c0 0x7fd5e4069d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd60415d7a0 tx=0x7fd5ec009040 comp rx=0 tx=0).stop 2024-05-04T12:19:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.917+0000 7fd609466640 1 -- 172.21.15.52:0/4185974390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd60406fa80 msgr2=0x7fd604152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.917+0000 7fd609466640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd60406fa80 0x7fd604152fd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400b2c0 tx=0x7fd5f401b900 comp rx=0 tx=0).stop 2024-05-04T12:19:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.917+0000 7fd609466640 1 -- 172.21.15.52:0/4185974390 shutdown_connections 2024-05-04T12:19:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.917+0000 7fd609466640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd5e40678c0 0x7fd5e4069d80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.917+0000 7fd609466640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd60415c170 0x7fd60415cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.917+0000 7fd609466640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd60415bd90 0x7fd604145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.917+0000 7fd609466640 1 --2- 172.21.15.52:0/4185974390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd60406fa80 0x7fd604152fd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.917+0000 7fd609466640 1 -- 172.21.15.52:0/4185974390 >> 172.21.15.52:0/4185974390 conn(0x7fd604067730 msgr2=0x7fd60406d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:04.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.917+0000 7fd609466640 1 -- 172.21.15.52:0/4185974390 shutdown_connections 2024-05-04T12:19:04.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:04.917+0000 7fd609466640 1 -- 172.21.15.52:0/4185974390 wait complete. 2024-05-04T12:19:04.939 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:19:05.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60ca121640 1 Processor -- start 2024-05-04T12:19:05.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60ca121640 1 -- start start 2024-05-04T12:19:05.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60ca121640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f60c4141010 0x7f60c40d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60ca121640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f60c4144570 0x7f60c4143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60ca121640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f60c40d2540 0x7f60c4143e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60ca121640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f60c40d29c0 con 0x7f60c4144570 2024-05-04T12:19:05.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60ca121640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f60c40d2ca0 con 0x7f60c40d2540 2024-05-04T12:19:05.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60ca121640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f60c40d2f80 con 0x7f60c4141010 2024-05-04T12:19:05.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60c2ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f60c4144570 0x7f60c4143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60c2ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f60c4144570 0x7f60c4143940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37900/0 (socket says 172.21.15.52:37900) 2024-05-04T12:19:05.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60c2ffd640 1 -- 172.21.15.52:0/2866576452 learned_addr learned my addr 172.21.15.52:0/2866576452 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:05.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60c37fe640 1 --2- 172.21.15.52:0/2866576452 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f60c4141010 0x7f60c40d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60c3fff640 1 --2- 172.21.15.52:0/2866576452 >> v2:172.21.15.111:3300/0 conn(0x7f60c40d2540 0x7f60c4143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60c2ffd640 1 -- 172.21.15.52:0/2866576452 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f60c4141010 msgr2=0x7f60c40d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:05.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60c2ffd640 1 --2- 172.21.15.52:0/2866576452 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f60c4141010 0x7f60c40d2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60c2ffd640 1 -- 172.21.15.52:0/2866576452 >> v2:172.21.15.111:3300/0 conn(0x7f60c40d2540 msgr2=0x7f60c4143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:05.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60c2ffd640 1 --2- 172.21.15.52:0/2866576452 >> v2:172.21.15.111:3300/0 conn(0x7f60c40d2540 0x7f60c4143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.061+0000 7f60c2ffd640 1 -- 172.21.15.52:0/2866576452 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c40d3260 con 0x7f60c4144570 2024-05-04T12:19:05.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60c2ffd640 1 --2- 172.21.15.52:0/2866576452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60c4144570 0x7f60c4143940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f60c4155880 tx=0x7f60b801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725369eb3b51de17 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:05.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60c27fc640 1 -- 172.21.15.52:0/2866576452 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b80ab640 con 0x7f60c4144570 2024-05-04T12:19:05.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60c27fc640 1 -- 172.21.15.52:0/2866576452 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b80b0070 con 0x7f60c4144570 2024-05-04T12:19:05.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60c27fc640 1 -- 172.21.15.52:0/2866576452 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b80aba00 con 0x7f60c4144570 2024-05-04T12:19:05.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 -- 172.21.15.52:0/2866576452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60c4144570 msgr2=0x7f60c4143940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 --2- 172.21.15.52:0/2866576452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60c4144570 0x7f60c4143940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f60c4155880 tx=0x7f60b801c460 comp rx=0 tx=0).stop 2024-05-04T12:19:05.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 -- 172.21.15.52:0/2866576452 shutdown_connections 2024-05-04T12:19:05.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 --2- 172.21.15.52:0/2866576452 >> v2:172.21.15.111:3300/0 conn(0x7f60c40d2540 0x7f60c4143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 --2- 172.21.15.52:0/2866576452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60c4144570 0x7f60c4143940 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 --2- 172.21.15.52:0/2866576452 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f60c4141010 0x7f60c40d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 -- 172.21.15.52:0/2866576452 >> 172.21.15.52:0/2866576452 conn(0x7f60c4072be0 msgr2=0x7f60c40d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:05.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 -- 172.21.15.52:0/2866576452 shutdown_connections 2024-05-04T12:19:05.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 -- 172.21.15.52:0/2866576452 wait complete. 2024-05-04T12:19:05.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 Processor -- start 2024-05-04T12:19:05.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 -- start start 2024-05-04T12:19:05.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60c40d2540 0x7f60c413ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f60c4141010 0x7f60c413b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60c4144570 0x7f60c4151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f60c4162990 con 0x7f60c4144570 2024-05-04T12:19:05.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f60c4162810 con 0x7f60c40d2540 2024-05-04T12:19:05.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.065+0000 7f60ca121640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f60c4162b10 con 0x7f60c4141010 2024-05-04T12:19:05.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60c3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60c4144570 0x7f60c4151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60c3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60c4144570 0x7f60c4151f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37916/0 (socket says 172.21.15.52:37916) 2024-05-04T12:19:05.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60c3fff640 1 -- 172.21.15.52:0/3361483267 learned_addr learned my addr 172.21.15.52:0/3361483267 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:05.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60c37fe640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60c40d2540 0x7f60c413ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60c2ffd640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f60c4141010 0x7f60c413b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60c3fff640 1 -- 172.21.15.52:0/3361483267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f60c4141010 msgr2=0x7f60c413b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60c3fff640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f60c4141010 0x7f60c413b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60c3fff640 1 -- 172.21.15.52:0/3361483267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60c40d2540 msgr2=0x7f60c413ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60c3fff640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60c40d2540 0x7f60c413ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60c3fff640 1 -- 172.21.15.52:0/3361483267 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c413cb40 con 0x7f60c4144570 2024-05-04T12:19:05.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60c2ffd640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f60c4141010 0x7f60c413b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:05.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60c3fff640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60c4144570 0x7f60c4151f20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f60bc007d40 tx=0x7f60bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:05.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60abfff640 1 -- 172.21.15.52:0/3361483267 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc014420 con 0x7f60c4144570 2024-05-04T12:19:05.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60abfff640 1 -- 172.21.15.52:0/3361483267 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60bc019070 con 0x7f60c4144570 2024-05-04T12:19:05.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60ca121640 1 -- 172.21.15.52:0/3361483267 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c413cde0 con 0x7f60c4144570 2024-05-04T12:19:05.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60abfff640 1 -- 172.21.15.52:0/3361483267 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc014800 con 0x7f60c4144570 2024-05-04T12:19:05.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60ca121640 1 -- 172.21.15.52:0/3361483267 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c4153970 con 0x7f60c4144570 2024-05-04T12:19:05.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60ca121640 1 -- 172.21.15.52:0/3361483267 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c4062e50 con 0x7f60c4144570 2024-05-04T12:19:05.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60abfff640 1 -- 172.21.15.52:0/3361483267 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f60bc0149a0 con 0x7f60c4144570 2024-05-04T12:19:05.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60abfff640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f60b00678c0 0x7f60b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60abfff640 1 -- 172.21.15.52:0/3361483267 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f60bc086490 con 0x7f60c4144570 2024-05-04T12:19:05.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60c37fe640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f60b00678c0 0x7f60b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.069+0000 7f60c37fe640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f60b00678c0 0x7f60b0069d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f60c415d880 tx=0x7f60b4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:05.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.073+0000 7f60abfff640 1 -- 172.21.15.52:0/3361483267 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f60bc056f40 con 0x7f60c4144570 2024-05-04T12:19:05.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.229+0000 7f60ca121640 1 -- 172.21.15.52:0/3361483267 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f60c4075320 con 0x7f60c4144570 2024-05-04T12:19:05.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.229+0000 7f60abfff640 1 -- 172.21.15.52:0/3361483267 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f60bc056840 con 0x7f60c4144570 2024-05-04T12:19:05.235 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:19:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.233+0000 7f60ca121640 1 -- 172.21.15.52:0/3361483267 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f60b00678c0 msgr2=0x7f60b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.233+0000 7f60ca121640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f60b00678c0 0x7f60b0069d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f60c415d880 tx=0x7f60b4009190 comp rx=0 tx=0).stop 2024-05-04T12:19:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.233+0000 7f60ca121640 1 -- 172.21.15.52:0/3361483267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60c4144570 msgr2=0x7f60c4151f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.233+0000 7f60ca121640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60c4144570 0x7f60c4151f20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f60bc007d40 tx=0x7f60bc00e4b0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.233+0000 7f60ca121640 1 -- 172.21.15.52:0/3361483267 shutdown_connections 2024-05-04T12:19:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.233+0000 7f60ca121640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f60b00678c0 0x7f60b0069d80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.233+0000 7f60ca121640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60c4144570 0x7f60c4151f20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.233+0000 7f60ca121640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f60c4141010 0x7f60c413b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.233+0000 7f60ca121640 1 --2- 172.21.15.52:0/3361483267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60c40d2540 0x7f60c413ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.233+0000 7f60ca121640 1 -- 172.21.15.52:0/3361483267 >> 172.21.15.52:0/3361483267 conn(0x7f60c4072be0 msgr2=0x7f60c41417b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.233+0000 7f60ca121640 1 -- 172.21.15.52:0/3361483267 shutdown_connections 2024-05-04T12:19:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.233+0000 7f60ca121640 1 -- 172.21.15.52:0/3361483267 wait complete. 2024-05-04T12:19:05.252 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-04T12:19:05.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.361+0000 7f80dc13f640 1 Processor -- start 2024-05-04T12:19:05.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.361+0000 7f80dc13f640 1 -- start start 2024-05-04T12:19:05.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80dc13f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f80d415bf50 0x7f80d415c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80dc13f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f80d40aa0c0 0x7f80d40ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80dc13f640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80d406d1b0 con 0x7f80d40aa550 2024-05-04T12:19:05.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80dc13f640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f80d406d490 con 0x7f80d40aa0c0 2024-05-04T12:19:05.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80dc13f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f80d406d770 con 0x7f80d415bf50 2024-05-04T12:19:05.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80d96da640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f80d40aa0c0 0x7f80d40ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80da6dc640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f80d40aa550 0x7f80d406e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:50012/0 (socket says 172.21.15.52:50012) 2024-05-04T12:19:05.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80da6dc640 1 -- 172.21.15.52:0/3431120149 learned_addr learned my addr 172.21.15.52:0/3431120149 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:05.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80d9edb640 1 --2- 172.21.15.52:0/3431120149 >> v2:172.21.15.111:3300/0 conn(0x7f80d415bf50 0x7f80d415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80d96da640 1 -- 172.21.15.52:0/3431120149 >> v2:172.21.15.111:3300/0 conn(0x7f80d415bf50 msgr2=0x7f80d415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:05.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80d96da640 1 --2- 172.21.15.52:0/3431120149 >> v2:172.21.15.111:3300/0 conn(0x7f80d415bf50 0x7f80d415c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80d96da640 1 -- 172.21.15.52:0/3431120149 >> v1:172.21.15.52:6789/0 conn(0x7f80d40aa550 legacy=0x7f80d406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80d8ed9640 1 -- 172.21.15.52:0/3431120149 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3693340391 0 0) 0x7f80d406d1b0 con 0x7f80d40aa550 2024-05-04T12:19:05.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80d96da640 1 -- 172.21.15.52:0/3431120149 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d406da50 con 0x7f80d40aa0c0 2024-05-04T12:19:05.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80d96da640 1 --2- 172.21.15.52:0/3431120149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80d40aa0c0 0x7f80d40ccb50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f80d4069260 tx=0x7f80c401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda3e33baeaa330f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:05.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80d8ed9640 1 -- 172.21.15.52:0/3431120149 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c40a7070 con 0x7f80d40aa0c0 2024-05-04T12:19:05.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80d8ed9640 1 -- 172.21.15.52:0/3431120149 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c40b0070 con 0x7f80d40aa0c0 2024-05-04T12:19:05.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.365+0000 7f80d8ed9640 1 -- 172.21.15.52:0/3431120149 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c40ab890 con 0x7f80d40aa0c0 2024-05-04T12:19:05.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 -- 172.21.15.52:0/3431120149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80d40aa0c0 msgr2=0x7f80d40ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 --2- 172.21.15.52:0/3431120149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80d40aa0c0 0x7f80d40ccb50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f80d4069260 tx=0x7f80c401c450 comp rx=0 tx=0).stop 2024-05-04T12:19:05.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 -- 172.21.15.52:0/3431120149 shutdown_connections 2024-05-04T12:19:05.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 --2- 172.21.15.52:0/3431120149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80d40aa0c0 0x7f80d40ccb50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 --2- 172.21.15.52:0/3431120149 >> v2:172.21.15.111:3300/0 conn(0x7f80d415bf50 0x7f80d415c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 -- 172.21.15.52:0/3431120149 >> 172.21.15.52:0/3431120149 conn(0x7f80d40b14a0 msgr2=0x7f80d40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:05.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 -- 172.21.15.52:0/3431120149 shutdown_connections 2024-05-04T12:19:05.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 -- 172.21.15.52:0/3431120149 wait complete. 2024-05-04T12:19:05.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 Processor -- start 2024-05-04T12:19:05.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 -- start start 2024-05-04T12:19:05.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80d40aa0c0 0x7f80d40b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80d40aa550 0x7f80d4148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80d415bf50 0x7f80d4157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4169030 con 0x7f80d40aa0c0 2024-05-04T12:19:05.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f80d40cd4a0 con 0x7f80d40aa550 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f80d41691b0 con 0x7f80d415bf50 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80d9edb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80d40aa0c0 0x7f80d40b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80d9edb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80d40aa0c0 0x7f80d40b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37934/0 (socket says 172.21.15.52:37934) 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80d9edb640 1 -- 172.21.15.52:0/828452295 learned_addr learned my addr 172.21.15.52:0/828452295 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80da6dc640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80d415bf50 0x7f80d4157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80d96da640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80d40aa550 0x7f80d4148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80d9edb640 1 -- 172.21.15.52:0/828452295 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80d415bf50 msgr2=0x7f80d4157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80d9edb640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80d415bf50 0x7f80d4157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80d9edb640 1 -- 172.21.15.52:0/828452295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80d40aa550 msgr2=0x7f80d4148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80d9edb640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80d40aa550 0x7f80d4148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80d9edb640 1 -- 172.21.15.52:0/828452295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d4148ff0 con 0x7f80d40aa0c0 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80da6dc640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80d415bf50 0x7f80d4157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80d9edb640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80d40aa0c0 0x7f80d40b2920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80d000d5f0 tx=0x7f80d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80ca7fc640 1 -- 172.21.15.52:0/828452295 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d00125f0 con 0x7f80d40aa0c0 2024-05-04T12:19:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80ca7fc640 1 -- 172.21.15.52:0/828452295 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d0019070 con 0x7f80d40aa0c0 2024-05-04T12:19:05.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 -- 172.21.15.52:0/828452295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d4153920 con 0x7f80d40aa0c0 2024-05-04T12:19:05.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80ca7fc640 1 -- 172.21.15.52:0/828452295 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d00129b0 con 0x7f80d40aa0c0 2024-05-04T12:19:05.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.369+0000 7f80dc13f640 1 -- 172.21.15.52:0/828452295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d4153ee0 con 0x7f80d40aa0c0 2024-05-04T12:19:05.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.373+0000 7f80dc13f640 1 -- 172.21.15.52:0/828452295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d4062e50 con 0x7f80d40aa0c0 2024-05-04T12:19:05.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.373+0000 7f80ca7fc640 1 -- 172.21.15.52:0/828452295 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f80d0012b50 con 0x7f80d40aa0c0 2024-05-04T12:19:05.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.373+0000 7f80ca7fc640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f80b80678c0 0x7f80b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.373+0000 7f80ca7fc640 1 -- 172.21.15.52:0/828452295 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f80d0086140 con 0x7f80d40aa0c0 2024-05-04T12:19:05.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.373+0000 7f80d96da640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f80b80678c0 0x7f80b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.373+0000 7f80d96da640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f80b80678c0 0x7f80b8069d80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f80c40af000 tx=0x7f80c40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:05.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.377+0000 7f80ca7fc640 1 -- 172.21.15.52:0/828452295 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f80d4148ff0 con 0x7f80d40aa0c0 2024-05-04T12:19:05.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.545+0000 7f80dc13f640 1 -- 172.21.15.52:0/828452295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f80d4075320 con 0x7f80d40aa0c0 2024-05-04T12:19:05.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.545+0000 7f80ca7fc640 1 -- 172.21.15.52:0/828452295 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f80d0011340 con 0x7f80d40aa0c0 2024-05-04T12:19:05.547 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-04T12:19:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.545+0000 7f80dc13f640 1 -- 172.21.15.52:0/828452295 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f80b80678c0 msgr2=0x7f80b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.545+0000 7f80dc13f640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f80b80678c0 0x7f80b8069d80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f80c40af000 tx=0x7f80c40a7000 comp rx=0 tx=0).stop 2024-05-04T12:19:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.545+0000 7f80dc13f640 1 -- 172.21.15.52:0/828452295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80d40aa0c0 msgr2=0x7f80d40b2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.545+0000 7f80dc13f640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80d40aa0c0 0x7f80d40b2920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80d000d5f0 tx=0x7f80d000dab0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.545+0000 7f80dc13f640 1 -- 172.21.15.52:0/828452295 shutdown_connections 2024-05-04T12:19:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.545+0000 7f80dc13f640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f80b80678c0 0x7f80b8069d80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.545+0000 7f80dc13f640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80d415bf50 0x7f80d4157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.545+0000 7f80dc13f640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80d40aa550 0x7f80d4148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.545+0000 7f80dc13f640 1 --2- 172.21.15.52:0/828452295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80d40aa0c0 0x7f80d40b2920 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.545+0000 7f80dc13f640 1 -- 172.21.15.52:0/828452295 >> 172.21.15.52:0/828452295 conn(0x7f80d40b14a0 msgr2=0x7f80d4159550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:05.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.545+0000 7f80dc13f640 1 -- 172.21.15.52:0/828452295 shutdown_connections 2024-05-04T12:19:05.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.545+0000 7f80dc13f640 1 -- 172.21.15.52:0/828452295 wait complete. 2024-05-04T12:19:05.565 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi052.front.sepia.ceph.com... 2024-05-04T12:19:05.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:05.565 DEBUG:teuthology.orchestra.run.smithi052:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-05-04T12:19:05.577 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T12:19:05.577 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T12:19:05.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:05.577 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-04T12:19:05.623 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:19:05.623 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T12:19:05.630 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:19:05.630 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T12:19:05.678 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:19:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.797+0000 7fe900c54640 1 Processor -- start 2024-05-04T12:19:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.797+0000 7fe900c54640 1 -- start start 2024-05-04T12:19:05.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.801+0000 7fe900c54640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8fc0cc690 0x7fe8fc0b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.801+0000 7fe900c54640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe8fc0b0ca0 0x7fe8fc070770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.801+0000 7fe900c54640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8fc06cf00 con 0x7fe8fc15e470 2024-05-04T12:19:05.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.801+0000 7fe900c54640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fe8fc06d1e0 con 0x7fe8fc0b0ca0 2024-05-04T12:19:05.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.801+0000 7fe900c54640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe8fc06d4c0 con 0x7fe8fc0cc690 2024-05-04T12:19:05.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.801+0000 7fe8fad9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe8fc0b0ca0 0x7fe8fc070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.801+0000 7fe8fad9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe8fc0b0ca0 0x7fe8fc070770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37936/0 (socket says 172.21.15.52:37936) 2024-05-04T12:19:05.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.801+0000 7fe8fad9d640 1 -- 172.21.15.52:0/1920976381 learned_addr learned my addr 172.21.15.52:0/1920976381 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:05.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.801+0000 7fe8f9d9b640 1 --2- 172.21.15.52:0/1920976381 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8fc0cc690 0x7fe8fc0b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.801+0000 7fe8fad9d640 1 -- 172.21.15.52:0/1920976381 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8fc0cc690 msgr2=0x7fe8fc0b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:05.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.801+0000 7fe8fad9d640 1 --2- 172.21.15.52:0/1920976381 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8fc0cc690 0x7fe8fc0b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.801+0000 7fe8fad9d640 1 -- 172.21.15.52:0/1920976381 >> v1:172.21.15.111:6789/0 conn(0x7fe8fc15e470 legacy=0x7fe8fc06e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:05.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.801+0000 7fe8fad9d640 1 -- 172.21.15.52:0/1920976381 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc06d7a0 con 0x7fe8fc0b0ca0 2024-05-04T12:19:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8fad9d640 1 --2- 172.21.15.52:0/1920976381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc0b0ca0 0x7fe8fc070770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009e50 tx=0x7fe8f401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc802580b040433 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8f959a640 1 -- 172.21.15.52:0/1920976381 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40a56f0 con 0x7fe8fc0b0ca0 2024-05-04T12:19:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8f959a640 1 -- 172.21.15.52:0/1920976381 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f40a5890 con 0x7fe8fc0b0ca0 2024-05-04T12:19:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8f959a640 1 -- 172.21.15.52:0/1920976381 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40a5c60 con 0x7fe8fc0b0ca0 2024-05-04T12:19:05.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 -- 172.21.15.52:0/1920976381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc0b0ca0 msgr2=0x7fe8fc070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 --2- 172.21.15.52:0/1920976381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc0b0ca0 0x7fe8fc070770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009e50 tx=0x7fe8f401c960 comp rx=0 tx=0).stop 2024-05-04T12:19:05.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 -- 172.21.15.52:0/1920976381 shutdown_connections 2024-05-04T12:19:05.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 --2- 172.21.15.52:0/1920976381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc0b0ca0 0x7fe8fc070770 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 --2- 172.21.15.52:0/1920976381 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8fc0cc690 0x7fe8fc0b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 -- 172.21.15.52:0/1920976381 >> 172.21.15.52:0/1920976381 conn(0x7fe8fc0b14a0 msgr2=0x7fe8fc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:05.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 -- 172.21.15.52:0/1920976381 shutdown_connections 2024-05-04T12:19:05.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 -- 172.21.15.52:0/1920976381 wait complete. 2024-05-04T12:19:05.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 Processor -- start 2024-05-04T12:19:05.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 -- start start 2024-05-04T12:19:05.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8fc0b0ca0 0x7fe8fc14ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8fc0cc690 0x7fe8fc14db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc15e470 0x7fe8fc155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc071850 con 0x7fe8fc15e470 2024-05-04T12:19:05.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc0716d0 con 0x7fe8fc0b0ca0 2024-05-04T12:19:05.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe900c54640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe8fc0719d0 con 0x7fe8fc0cc690 2024-05-04T12:19:05.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8fad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc15e470 0x7fe8fc155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8fad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc15e470 0x7fe8fc155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37952/0 (socket says 172.21.15.52:37952) 2024-05-04T12:19:05.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8fad9d640 1 -- 172.21.15.52:0/4171729338 learned_addr learned my addr 172.21.15.52:0/4171729338 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:05.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8f9d9b640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8fc0cc690 0x7fe8fc14db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8fa59c640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8fc0b0ca0 0x7fe8fc14ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8fad9d640 1 -- 172.21.15.52:0/4171729338 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8fc0cc690 msgr2=0x7fe8fc14db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8fad9d640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8fc0cc690 0x7fe8fc14db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8fad9d640 1 -- 172.21.15.52:0/4171729338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8fc0b0ca0 msgr2=0x7fe8fc14ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8fad9d640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8fc0b0ca0 0x7fe8fc14ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8fad9d640 1 -- 172.21.15.52:0/4171729338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc14e090 con 0x7fe8fc15e470 2024-05-04T12:19:05.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8fa59c640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8fc0b0ca0 0x7fe8fc14ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:05.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.805+0000 7fe8fad9d640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc15e470 0x7fe8fc155520 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40af040 tx=0x7fe8f4019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:05.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.809+0000 7fe8eaffd640 1 -- 172.21.15.52:0/4171729338 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40bc020 con 0x7fe8fc15e470 2024-05-04T12:19:05.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.809+0000 7fe8eaffd640 1 -- 172.21.15.52:0/4171729338 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f40b88a0 con 0x7fe8fc15e470 2024-05-04T12:19:05.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.809+0000 7fe900c54640 1 -- 172.21.15.52:0/4171729338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc14f3b0 con 0x7fe8fc15e470 2024-05-04T12:19:05.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.809+0000 7fe8eaffd640 1 -- 172.21.15.52:0/4171729338 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40b8c90 con 0x7fe8fc15e470 2024-05-04T12:19:05.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.809+0000 7fe900c54640 1 -- 172.21.15.52:0/4171729338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc158880 con 0x7fe8fc15e470 2024-05-04T12:19:05.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.809+0000 7fe900c54640 1 -- 172.21.15.52:0/4171729338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8fc062e50 con 0x7fe8fc15e470 2024-05-04T12:19:05.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.809+0000 7fe8eaffd640 1 -- 172.21.15.52:0/4171729338 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe8f40bf020 con 0x7fe8fc15e470 2024-05-04T12:19:05.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.809+0000 7fe8eaffd640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe8dc0678f0 0x7fe8dc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:05.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.809+0000 7fe8eaffd640 1 -- 172.21.15.52:0/4171729338 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7fe8f4121150 con 0x7fe8fc15e470 2024-05-04T12:19:05.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.809+0000 7fe8fa59c640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe8dc0678f0 0x7fe8dc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:05.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.809+0000 7fe8fa59c640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe8dc0678f0 0x7fe8dc069db0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0098b0 tx=0x7fe8ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:05.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.813+0000 7fe8eaffd640 1 -- 172.21.15.52:0/4171729338 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe8f40165e0 con 0x7fe8fc15e470 2024-05-04T12:19:05.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.969+0000 7fe900c54640 1 -- 172.21.15.52:0/4171729338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8fc075320 con 0x7fe8fc15e470 2024-05-04T12:19:05.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.973+0000 7fe8eaffd640 1 -- 172.21.15.52:0/4171729338 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16913 (secure 0 0 0) 0x7fe8f40f1c10 con 0x7fe8fc15e470 2024-05-04T12:19:05.975 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:05.975 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:19:04.046290+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:19:05.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.973+0000 7fe900c54640 1 -- 172.21.15.52:0/4171729338 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe8dc0678f0 msgr2=0x7fe8dc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.973+0000 7fe900c54640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe8dc0678f0 0x7fe8dc069db0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0098b0 tx=0x7fe8ec008040 comp rx=0 tx=0).stop 2024-05-04T12:19:05.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.973+0000 7fe900c54640 1 -- 172.21.15.52:0/4171729338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc15e470 msgr2=0x7fe8fc155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:05.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.973+0000 7fe900c54640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc15e470 0x7fe8fc155520 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40af040 tx=0x7fe8f4019700 comp rx=0 tx=0).stop 2024-05-04T12:19:05.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.973+0000 7fe900c54640 1 -- 172.21.15.52:0/4171729338 shutdown_connections 2024-05-04T12:19:05.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.973+0000 7fe900c54640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe8dc0678f0 0x7fe8dc069db0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.973+0000 7fe900c54640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc15e470 0x7fe8fc155520 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.973+0000 7fe900c54640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8fc0cc690 0x7fe8fc14db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.973+0000 7fe900c54640 1 --2- 172.21.15.52:0/4171729338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8fc0b0ca0 0x7fe8fc14ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:05.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.973+0000 7fe900c54640 1 -- 172.21.15.52:0/4171729338 >> 172.21.15.52:0/4171729338 conn(0x7fe8fc0b14a0 msgr2=0x7fe8fc153960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:05.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.973+0000 7fe900c54640 1 -- 172.21.15.52:0/4171729338 shutdown_connections 2024-05-04T12:19:05.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:05.973+0000 7fe900c54640 1 -- 172.21.15.52:0/4171729338 wait complete. 2024-05-04T12:19:05.992 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:19:05.993 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:19:06.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.105+0000 7f58e4439640 1 Processor -- start 2024-05-04T12:19:06.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.105+0000 7f58e4439640 1 -- start start 2024-05-04T12:19:06.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.105+0000 7f58e4439640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58dc145960 0x7f58dc160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.105+0000 7f58e4439640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f58dc15c590 0x7f58dc0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.105+0000 7f58e4439640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f58dc0cccd0 0x7f58dc0cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.105+0000 7f58e4439640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f58dc0cd140 con 0x7f58dc0cccd0 2024-05-04T12:19:06.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.105+0000 7f58e4439640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f58dc06dbf0 con 0x7f58dc15c590 2024-05-04T12:19:06.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.105+0000 7f58e4439640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f58dc160ad0 con 0x7f58dc145960 2024-05-04T12:19:06.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e29d6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f58dc0cccd0 0x7f58dc0cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e21d5640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58dc145960 0x7f58dc160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e29d6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f58dc0cccd0 0x7f58dc0cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37954/0 (socket says 172.21.15.52:37954) 2024-05-04T12:19:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e29d6640 1 -- 172.21.15.52:0/1811901159 learned_addr learned my addr 172.21.15.52:0/1811901159 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e19d4640 1 --2- 172.21.15.52:0/1811901159 >> v2:172.21.15.111:3300/0 conn(0x7f58dc15c590 0x7f58dc0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e29d6640 1 -- 172.21.15.52:0/1811901159 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58dc145960 msgr2=0x7f58dc160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e29d6640 1 --2- 172.21.15.52:0/1811901159 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58dc145960 0x7f58dc160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e29d6640 1 -- 172.21.15.52:0/1811901159 >> v2:172.21.15.111:3300/0 conn(0x7f58dc15c590 msgr2=0x7f58dc0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e29d6640 1 --2- 172.21.15.52:0/1811901159 >> v2:172.21.15.111:3300/0 conn(0x7f58dc15c590 0x7f58dc0cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e29d6640 1 -- 172.21.15.52:0/1811901159 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc06cfe0 con 0x7f58dc0cccd0 2024-05-04T12:19:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e29d6640 1 --2- 172.21.15.52:0/1811901159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc0cccd0 0x7f58dc0cdf50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0b2470 tx=0x7f58d401c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8eb8ae03c2a7564 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e11d3640 1 -- 172.21.15.52:0/1811901159 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d40a6500 con 0x7f58dc0cccd0 2024-05-04T12:19:06.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e11d3640 1 -- 172.21.15.52:0/1811901159 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d40a66a0 con 0x7f58dc0cccd0 2024-05-04T12:19:06.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e11d3640 1 -- 172.21.15.52:0/1811901159 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d40a6a70 con 0x7f58dc0cccd0 2024-05-04T12:19:06.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e4439640 1 -- 172.21.15.52:0/1811901159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc0cccd0 msgr2=0x7f58dc0cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e4439640 1 --2- 172.21.15.52:0/1811901159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc0cccd0 0x7f58dc0cdf50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0b2470 tx=0x7f58d401c670 comp rx=0 tx=0).stop 2024-05-04T12:19:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e4439640 1 -- 172.21.15.52:0/1811901159 shutdown_connections 2024-05-04T12:19:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e4439640 1 --2- 172.21.15.52:0/1811901159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc0cccd0 0x7f58dc0cdf50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e4439640 1 --2- 172.21.15.52:0/1811901159 >> v2:172.21.15.111:3300/0 conn(0x7f58dc15c590 0x7f58dc0cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e4439640 1 --2- 172.21.15.52:0/1811901159 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58dc145960 0x7f58dc160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e4439640 1 -- 172.21.15.52:0/1811901159 >> 172.21.15.52:0/1811901159 conn(0x7f58dc067730 msgr2=0x7f58dc0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e4439640 1 -- 172.21.15.52:0/1811901159 shutdown_connections 2024-05-04T12:19:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.109+0000 7f58e4439640 1 -- 172.21.15.52:0/1811901159 wait complete. 2024-05-04T12:19:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e4439640 1 Processor -- start 2024-05-04T12:19:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e4439640 1 -- start start 2024-05-04T12:19:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e4439640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58dc0cccd0 0x7f58dc159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e4439640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58dc145960 0x7f58dc14a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e4439640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc15c590 0x7f58dc139510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e4439640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc06cc40 con 0x7f58dc15c590 2024-05-04T12:19:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e4439640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc0cf890 con 0x7f58dc145960 2024-05-04T12:19:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e4439640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f58dc06cdc0 con 0x7f58dc0cccd0 2024-05-04T12:19:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e29d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc15c590 0x7f58dc139510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e29d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc15c590 0x7f58dc139510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37956/0 (socket says 172.21.15.52:37956) 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e29d6640 1 -- 172.21.15.52:0/743106358 learned_addr learned my addr 172.21.15.52:0/743106358 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e21d5640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58dc0cccd0 0x7f58dc159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e19d4640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58dc145960 0x7f58dc14a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e29d6640 1 -- 172.21.15.52:0/743106358 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58dc0cccd0 msgr2=0x7f58dc159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e29d6640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58dc0cccd0 0x7f58dc159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e29d6640 1 -- 172.21.15.52:0/743106358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58dc145960 msgr2=0x7f58dc14a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e29d6640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58dc145960 0x7f58dc14a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e29d6640 1 -- 172.21.15.52:0/743106358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc139c80 con 0x7f58dc15c590 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e19d4640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58dc145960 0x7f58dc14a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e29d6640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc15c590 0x7f58dc139510 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f58d401cf10 tx=0x7f58d4019410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58d2ffd640 1 -- 172.21.15.52:0/743106358 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d40a6a50 con 0x7f58dc15c590 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58d2ffd640 1 -- 172.21.15.52:0/743106358 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d40a6bf0 con 0x7f58dc15c590 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e4439640 1 -- 172.21.15.52:0/743106358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc139f60 con 0x7f58dc15c590 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58d2ffd640 1 -- 172.21.15.52:0/743106358 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d40af480 con 0x7f58dc15c590 2024-05-04T12:19:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e4439640 1 -- 172.21.15.52:0/743106358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc0afc30 con 0x7f58dc15c590 2024-05-04T12:19:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.113+0000 7f58e4439640 1 -- 172.21.15.52:0/743106358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58dc062e50 con 0x7f58dc15c590 2024-05-04T12:19:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.117+0000 7f58d2ffd640 1 -- 172.21.15.52:0/743106358 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f58d40af620 con 0x7f58dc15c590 2024-05-04T12:19:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.117+0000 7f58d2ffd640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f58c00678c0 0x7f58c0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.117+0000 7f58d2ffd640 1 -- 172.21.15.52:0/743106358 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f58d41201e0 con 0x7f58dc15c590 2024-05-04T12:19:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.117+0000 7f58e21d5640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f58c00678c0 0x7f58c0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.117+0000 7f58e21d5640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f58c00678c0 0x7f58c0069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009af0 tx=0x7f58d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:06.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.121+0000 7f58d2ffd640 1 -- 172.21.15.52:0/743106358 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f58d40165e0 con 0x7f58dc15c590 2024-05-04T12:19:06.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.281+0000 7f58e4439640 1 -- 172.21.15.52:0/743106358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f58dc132000 con 0x7f58dc15c590 2024-05-04T12:19:06.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.281+0000 7f58d2ffd640 1 -- 172.21.15.52:0/743106358 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f58d40f0c90 con 0x7f58dc15c590 2024-05-04T12:19:06.284 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:19:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.281+0000 7f58e4439640 1 -- 172.21.15.52:0/743106358 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f58c00678c0 msgr2=0x7f58c0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.281+0000 7f58e4439640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f58c00678c0 0x7f58c0069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009af0 tx=0x7f58d8008040 comp rx=0 tx=0).stop 2024-05-04T12:19:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.281+0000 7f58e4439640 1 -- 172.21.15.52:0/743106358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc15c590 msgr2=0x7f58dc139510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.281+0000 7f58e4439640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc15c590 0x7f58dc139510 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f58d401cf10 tx=0x7f58d4019410 comp rx=0 tx=0).stop 2024-05-04T12:19:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.281+0000 7f58e4439640 1 -- 172.21.15.52:0/743106358 shutdown_connections 2024-05-04T12:19:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.281+0000 7f58e4439640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f58c00678c0 0x7f58c0069d80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.281+0000 7f58e4439640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc15c590 0x7f58dc139510 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.281+0000 7f58e4439640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58dc145960 0x7f58dc14a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.281+0000 7f58e4439640 1 --2- 172.21.15.52:0/743106358 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58dc0cccd0 0x7f58dc159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.281+0000 7f58e4439640 1 -- 172.21.15.52:0/743106358 >> 172.21.15.52:0/743106358 conn(0x7f58dc067730 msgr2=0x7f58dc06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:06.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.285+0000 7f58e4439640 1 -- 172.21.15.52:0/743106358 shutdown_connections 2024-05-04T12:19:06.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.285+0000 7f58e4439640 1 -- 172.21.15.52:0/743106358 wait complete. 2024-05-04T12:19:06.304 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:19:06.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f2569611640 1 Processor -- start 2024-05-04T12:19:06.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f2569611640 1 -- start start 2024-05-04T12:19:06.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f2569611640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2564144570 0x7f2564143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f2569611640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2564143e80 0x7f25641418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f2569611640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25640d2710 con 0x7f2564141010 2024-05-04T12:19:06.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f2569611640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f25640d29c0 con 0x7f2564143e80 2024-05-04T12:19:06.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f2569611640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f25640d2ca0 con 0x7f2564144570 2024-05-04T12:19:06.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f25637fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2564143e80 0x7f25641418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f2562ffd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f2564141010 0x7f25640d2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46810/0 (socket says 172.21.15.52:46810) 2024-05-04T12:19:06.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f2562ffd640 1 -- 172.21.15.52:0/2968688385 learned_addr learned my addr 172.21.15.52:0/2968688385 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:06.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f25627fc640 1 --2- 172.21.15.52:0/2968688385 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2564144570 0x7f2564143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f25637fe640 1 -- 172.21.15.52:0/2968688385 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2564144570 msgr2=0x7f2564143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:06.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f25637fe640 1 --2- 172.21.15.52:0/2968688385 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2564144570 0x7f2564143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f25637fe640 1 -- 172.21.15.52:0/2968688385 >> v1:172.21.15.52:6789/0 conn(0x7f2564141010 legacy=0x7f25640d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f2561ffb640 1 -- 172.21.15.52:0/2968688385 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2349586477 0 0) 0x7f25640d2710 con 0x7f2564141010 2024-05-04T12:19:06.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.429+0000 7f25637fe640 1 -- 172.21.15.52:0/2968688385 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25640d2f80 con 0x7f2564143e80 2024-05-04T12:19:06.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f25637fe640 1 --2- 172.21.15.52:0/2968688385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2564143e80 0x7f25641418b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f255c009d90 tx=0x7f255c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec88541c22a1bf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:06.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2561ffb640 1 -- 172.21.15.52:0/2968688385 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c0b2070 con 0x7f2564143e80 2024-05-04T12:19:06.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2561ffb640 1 -- 172.21.15.52:0/2968688385 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f255c0a5890 con 0x7f2564143e80 2024-05-04T12:19:06.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2561ffb640 1 -- 172.21.15.52:0/2968688385 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c0a5c60 con 0x7f2564143e80 2024-05-04T12:19:06.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 -- 172.21.15.52:0/2968688385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2564143e80 msgr2=0x7f25641418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 --2- 172.21.15.52:0/2968688385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2564143e80 0x7f25641418b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f255c009d90 tx=0x7f255c01c8e0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 -- 172.21.15.52:0/2968688385 shutdown_connections 2024-05-04T12:19:06.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 --2- 172.21.15.52:0/2968688385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2564143e80 0x7f25641418b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 --2- 172.21.15.52:0/2968688385 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2564144570 0x7f2564143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 -- 172.21.15.52:0/2968688385 >> 172.21.15.52:0/2968688385 conn(0x7f2564072be0 msgr2=0x7f25640d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:06.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 -- 172.21.15.52:0/2968688385 shutdown_connections 2024-05-04T12:19:06.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 -- 172.21.15.52:0/2968688385 wait complete. 2024-05-04T12:19:06.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 Processor -- start 2024-05-04T12:19:06.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 -- start start 2024-05-04T12:19:06.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2564141010 0x7f25641582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2564143e80 0x7f2564138c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2564144570 0x7f25640aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2564142780 con 0x7f2564143e80 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2564142600 con 0x7f2564141010 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2569611640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f2564142900 con 0x7f2564144570 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f25627fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2564143e80 0x7f2564138c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f25627fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2564143e80 0x7f2564138c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56450/0 (socket says 172.21.15.52:56450) 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f25627fc640 1 -- 172.21.15.52:0/2385738183 learned_addr learned my addr 172.21.15.52:0/2385738183 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2562ffd640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2564141010 0x7f25641582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f25637fe640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2564144570 0x7f25640aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f25627fc640 1 -- 172.21.15.52:0/2385738183 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2564144570 msgr2=0x7f25640aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f25627fc640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2564144570 0x7f25640aa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f25627fc640 1 -- 172.21.15.52:0/2385738183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2564141010 msgr2=0x7f25641582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f25627fc640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2564141010 0x7f25641582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f25627fc640 1 -- 172.21.15.52:0/2385738183 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25641393c0 con 0x7f2564143e80 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.433+0000 7f2562ffd640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2564141010 0x7f25641582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.437+0000 7f25627fc640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2564143e80 0x7f2564138c70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f255800d7d0 tx=0x7f255800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.437+0000 7f254b7fe640 1 -- 172.21.15.52:0/2385738183 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255800b6c0 con 0x7f2564143e80 2024-05-04T12:19:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.437+0000 7f254b7fe640 1 -- 172.21.15.52:0/2385738183 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2558019070 con 0x7f2564143e80 2024-05-04T12:19:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.437+0000 7f2569611640 1 -- 172.21.15.52:0/2385738183 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25641396a0 con 0x7f2564143e80 2024-05-04T12:19:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.437+0000 7f254b7fe640 1 -- 172.21.15.52:0/2385738183 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255800bac0 con 0x7f2564143e80 2024-05-04T12:19:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.437+0000 7f2569611640 1 -- 172.21.15.52:0/2385738183 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2564139c60 con 0x7f2564143e80 2024-05-04T12:19:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.437+0000 7f2569611640 1 -- 172.21.15.52:0/2385738183 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2564062e50 con 0x7f2564143e80 2024-05-04T12:19:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.437+0000 7f254b7fe640 1 -- 172.21.15.52:0/2385738183 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f255800bc60 con 0x7f2564143e80 2024-05-04T12:19:06.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.437+0000 7f254b7fe640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f25500678f0 0x7f2550069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.437+0000 7f254b7fe640 1 -- 172.21.15.52:0/2385738183 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f2558086290 con 0x7f2564143e80 2024-05-04T12:19:06.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.437+0000 7f2562ffd640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f25500678f0 0x7f2550069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.437+0000 7f2562ffd640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f25500678f0 0x7f2550069db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2554000f70 tx=0x7f2554009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:06.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.441+0000 7f254b7fe640 1 -- 172.21.15.52:0/2385738183 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2558056ca0 con 0x7f2564143e80 2024-05-04T12:19:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.601+0000 7f2569611640 1 -- 172.21.15.52:0/2385738183 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f25640d5470 con 0x7f2564143e80 2024-05-04T12:19:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.601+0000 7f254b7fe640 1 -- 172.21.15.52:0/2385738183 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f25580565a0 con 0x7f2564143e80 2024-05-04T12:19:06.603 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:19:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.601+0000 7f2569611640 1 -- 172.21.15.52:0/2385738183 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f25500678f0 msgr2=0x7f2550069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.601+0000 7f2569611640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f25500678f0 0x7f2550069db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2554000f70 tx=0x7f2554009040 comp rx=0 tx=0).stop 2024-05-04T12:19:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.601+0000 7f2569611640 1 -- 172.21.15.52:0/2385738183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2564143e80 msgr2=0x7f2564138c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.601+0000 7f2569611640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2564143e80 0x7f2564138c70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f255800d7d0 tx=0x7f255800dc90 comp rx=0 tx=0).stop 2024-05-04T12:19:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.601+0000 7f2569611640 1 -- 172.21.15.52:0/2385738183 shutdown_connections 2024-05-04T12:19:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.601+0000 7f2569611640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f25500678f0 0x7f2550069db0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.601+0000 7f2569611640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2564144570 0x7f25640aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.601+0000 7f2569611640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2564143e80 0x7f2564138c70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.601+0000 7f2569611640 1 --2- 172.21.15.52:0/2385738183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2564141010 0x7f25641582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.601+0000 7f2569611640 1 -- 172.21.15.52:0/2385738183 >> 172.21.15.52:0/2385738183 conn(0x7f2564072be0 msgr2=0x7f256415c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.601+0000 7f2569611640 1 -- 172.21.15.52:0/2385738183 shutdown_connections 2024-05-04T12:19:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.601+0000 7f2569611640 1 -- 172.21.15.52:0/2385738183 wait complete. 2024-05-04T12:19:06.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:19:06.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa278d4c640 1 Processor -- start 2024-05-04T12:19:06.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa278d4c640 1 -- start start 2024-05-04T12:19:06.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa278d4c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa27415c590 0x7fa2740cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa278d4c640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa27406dbf0 con 0x7fa274145960 2024-05-04T12:19:06.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa278d4c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa27406cfe0 con 0x7fa2740cccd0 2024-05-04T12:19:06.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa278d4c640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa27406d2c0 con 0x7fa27415c590 2024-05-04T12:19:06.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27259c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa274145960 0x7fa274160590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46816/0 (socket says 172.21.15.52:46816) 2024-05-04T12:19:06.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27259c640 1 -- 172.21.15.52:0/1511492805 learned_addr learned my addr 172.21.15.52:0/1511492805 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa271d9b640 1 --2- 172.21.15.52:0/1511492805 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa27415c590 0x7fa2740cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 -- 172.21.15.52:0/1511492805 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2576140519 0 0) 0x7fa27406dbf0 con 0x7fa274145960 2024-05-04T12:19:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 -- 172.21.15.52:0/1511492805 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa25c003530 con 0x7fa274145960 2024-05-04T12:19:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 -- 172.21.15.52:0/1511492805 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2747967007 0 0) 0x7fa25c003530 con 0x7fa274145960 2024-05-04T12:19:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 -- 172.21.15.52:0/1511492805 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa27415c590 msgr2=0x7fa2740cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 --2- 172.21.15.52:0/1511492805 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa27415c590 0x7fa2740cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 -- 172.21.15.52:0/1511492805 >> v1:172.21.15.111:6789/0 conn(0x7fa2740cccd0 legacy=0x7fa27406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 -- 172.21.15.52:0/1511492805 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27406d5a0 con 0x7fa274145960 2024-05-04T12:19:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 -- 172.21.15.52:0/1511492805 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7fa264001680 con 0x7fa274145960 2024-05-04T12:19:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 -- 172.21.15.52:0/1511492805 >> v1:172.21.15.52:6789/0 conn(0x7fa274145960 legacy=0x7fa274160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 --2- 172.21.15.52:0/1511492805 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa27415c590 0x7fa25c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 --2- 172.21.15.52:0/1511492805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c003f70 0x7fa25c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 --2- 172.21.15.52:0/1511492805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa25c004d50 0x7fa25c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 -- 172.21.15.52:0/1511492805 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa27406d2c0 con 0x7fa25c004d50 2024-05-04T12:19:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 -- 172.21.15.52:0/1511492805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c00bb60 con 0x7fa25c003f70 2024-05-04T12:19:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27159a640 1 -- 172.21.15.52:0/1511492805 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa25c00bd60 con 0x7fa27415c590 2024-05-04T12:19:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa272d9d640 1 --2- 172.21.15.52:0/1511492805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa25c004d50 0x7fa25c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa271d9b640 1 --2- 172.21.15.52:0/1511492805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c003f70 0x7fa25c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa27259c640 1 --2- 172.21.15.52:0/1511492805 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa27415c590 0x7fa25c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa272d9d640 1 -- 172.21.15.52:0/1511492805 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa27415c590 msgr2=0x7fa25c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:06.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa272d9d640 1 --2- 172.21.15.52:0/1511492805 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa27415c590 0x7fa25c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa272d9d640 1 -- 172.21.15.52:0/1511492805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c003f70 msgr2=0x7fa25c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:06.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa272d9d640 1 --2- 172.21.15.52:0/1511492805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c003f70 0x7fa25c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.745+0000 7fa272d9d640 1 -- 172.21.15.52:0/1511492805 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25c003530 con 0x7fa25c004d50 2024-05-04T12:19:06.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa272d9d640 1 --2- 172.21.15.52:0/1511492805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa25c004d50 0x7fa25c0051d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa26c001e70 tx=0x7fa26c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c42cc1399e67f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:06.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa27159a640 1 -- 172.21.15.52:0/1511492805 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c001a90 con 0x7fa25c004d50 2024-05-04T12:19:06.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa27159a640 1 -- 172.21.15.52:0/1511492805 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa26c003b80 con 0x7fa25c004d50 2024-05-04T12:19:06.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa27159a640 1 -- 172.21.15.52:0/1511492805 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c005080 con 0x7fa25c004d50 2024-05-04T12:19:06.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa278d4c640 1 -- 172.21.15.52:0/1511492805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa25c004d50 msgr2=0x7fa25c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa278d4c640 1 --2- 172.21.15.52:0/1511492805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa25c004d50 0x7fa25c0051d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa26c001e70 tx=0x7fa26c01c780 comp rx=0 tx=0).stop 2024-05-04T12:19:06.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa272d9d640 1 -- 172.21.15.52:0/1511492805 reap_dead start 2024-05-04T12:19:06.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa278d4c640 1 -- 172.21.15.52:0/1511492805 shutdown_connections 2024-05-04T12:19:06.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa278d4c640 1 -- 172.21.15.52:0/1511492805 >> 172.21.15.52:0/1511492805 conn(0x7fa274067730 msgr2=0x7fa2740b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:06.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa278d4c640 1 -- 172.21.15.52:0/1511492805 shutdown_connections 2024-05-04T12:19:06.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa278d4c640 1 -- 172.21.15.52:0/1511492805 wait complete. 2024-05-04T12:19:06.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa278d4c640 1 Processor -- start 2024-05-04T12:19:06.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa278d4c640 1 -- start start 2024-05-04T12:19:06.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa278d4c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa27413a040 0x7fa274137130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa278d4c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2741f5140 0x7fa2740ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa278d4c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2740ae3b0 0x7fa2741f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa278d4c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa27406cc40 con 0x7fa2740ae3b0 2024-05-04T12:19:06.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa278d4c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa2740cf890 con 0x7fa27413a040 2024-05-04T12:19:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa278d4c640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa27406cdc0 con 0x7fa2741f5140 2024-05-04T12:19:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa272d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2740ae3b0 0x7fa2741f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa272d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2740ae3b0 0x7fa2741f7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56472/0 (socket says 172.21.15.52:56472) 2024-05-04T12:19:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa272d9d640 1 -- 172.21.15.52:0/2923683697 learned_addr learned my addr 172.21.15.52:0/2923683697 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa271d9b640 1 --2- 172.21.15.52:0/2923683697 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2741f5140 0x7fa2740ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa272d9d640 1 -- 172.21.15.52:0/2923683697 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2741f5140 msgr2=0x7fa2740ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa272d9d640 1 --2- 172.21.15.52:0/2923683697 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2741f5140 0x7fa2740ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa272d9d640 1 -- 172.21.15.52:0/2923683697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa27413a040 msgr2=0x7fa274137130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa27259c640 1 --2- 172.21.15.52:0/2923683697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa27413a040 0x7fa274137130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa272d9d640 1 --2- 172.21.15.52:0/2923683697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa27413a040 0x7fa274137130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.749+0000 7fa272d9d640 1 -- 172.21.15.52:0/2923683697 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa274137d70 con 0x7fa2740ae3b0 2024-05-04T12:19:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.753+0000 7fa272d9d640 1 --2- 172.21.15.52:0/2923683697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2740ae3b0 0x7fa2741f7ec0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa26c098080 tx=0x7fa25c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.753+0000 7fa262ffd640 1 -- 172.21.15.52:0/2923683697 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c007dc0 con 0x7fa2740ae3b0 2024-05-04T12:19:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.753+0000 7fa262ffd640 1 -- 172.21.15.52:0/2923683697 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa26c005820 con 0x7fa2740ae3b0 2024-05-04T12:19:06.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.753+0000 7fa278d4c640 1 -- 172.21.15.52:0/2923683697 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa274138050 con 0x7fa2740ae3b0 2024-05-04T12:19:06.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.753+0000 7fa262ffd640 1 -- 172.21.15.52:0/2923683697 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c001a90 con 0x7fa2740ae3b0 2024-05-04T12:19:06.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.753+0000 7fa278d4c640 1 -- 172.21.15.52:0/2923683697 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27413a420 con 0x7fa2740ae3b0 2024-05-04T12:19:06.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.753+0000 7fa278d4c640 1 -- 172.21.15.52:0/2923683697 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa274062e50 con 0x7fa2740ae3b0 2024-05-04T12:19:06.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.753+0000 7fa262ffd640 1 -- 172.21.15.52:0/2923683697 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa26c0be470 con 0x7fa2740ae3b0 2024-05-04T12:19:06.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.753+0000 7fa262ffd640 1 --2- 172.21.15.52:0/2923683697 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa2540678f0 0x7fa254069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:06.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.753+0000 7fa262ffd640 1 -- 172.21.15.52:0/2923683697 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7fa26c11feb0 con 0x7fa2740ae3b0 2024-05-04T12:19:06.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.753+0000 7fa27259c640 1 --2- 172.21.15.52:0/2923683697 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa2540678f0 0x7fa254069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:06.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.753+0000 7fa27259c640 1 --2- 172.21.15.52:0/2923683697 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa2540678f0 0x7fa254069db0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa264004850 tx=0x7fa264001c40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:06.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.757+0000 7fa262ffd640 1 -- 172.21.15.52:0/2923683697 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa26c0ad020 con 0x7fa2740ae3b0 2024-05-04T12:19:06.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.913+0000 7fa278d4c640 1 -- 172.21.15.52:0/2923683697 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa274075320 con 0x7fa2740ae3b0 2024-05-04T12:19:06.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.913+0000 7fa262ffd640 1 -- 172.21.15.52:0/2923683697 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fa26c016490 con 0x7fa2740ae3b0 2024-05-04T12:19:06.919 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:19:06.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.917+0000 7fa278d4c640 1 -- 172.21.15.52:0/2923683697 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa2540678f0 msgr2=0x7fa254069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.917+0000 7fa278d4c640 1 --2- 172.21.15.52:0/2923683697 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa2540678f0 0x7fa254069db0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa264004850 tx=0x7fa264001c40 comp rx=0 tx=0).stop 2024-05-04T12:19:06.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.917+0000 7fa278d4c640 1 -- 172.21.15.52:0/2923683697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2740ae3b0 msgr2=0x7fa2741f7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:06.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.917+0000 7fa278d4c640 1 --2- 172.21.15.52:0/2923683697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2740ae3b0 0x7fa2741f7ec0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa26c098080 tx=0x7fa25c005840 comp rx=0 tx=0).stop 2024-05-04T12:19:06.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.917+0000 7fa278d4c640 1 -- 172.21.15.52:0/2923683697 shutdown_connections 2024-05-04T12:19:06.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.917+0000 7fa278d4c640 1 --2- 172.21.15.52:0/2923683697 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa2540678f0 0x7fa254069db0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.917+0000 7fa278d4c640 1 --2- 172.21.15.52:0/2923683697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2740ae3b0 0x7fa2741f7ec0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.917+0000 7fa278d4c640 1 --2- 172.21.15.52:0/2923683697 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2741f5140 0x7fa2740ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.917+0000 7fa278d4c640 1 --2- 172.21.15.52:0/2923683697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa27413a040 0x7fa274137130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:06.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.917+0000 7fa278d4c640 1 -- 172.21.15.52:0/2923683697 >> 172.21.15.52:0/2923683697 conn(0x7fa274067730 msgr2=0x7fa274140080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:06.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.917+0000 7fa278d4c640 1 -- 172.21.15.52:0/2923683697 shutdown_connections 2024-05-04T12:19:06.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:06.917+0000 7fa278d4c640 1 -- 172.21.15.52:0/2923683697 wait complete. 2024-05-04T12:19:06.937 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-04T12:19:07.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f248cf42640 1 Processor -- start 2024-05-04T12:19:07.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f248cf42640 1 -- start start 2024-05-04T12:19:07.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f248cf42640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f24880aa0c0 0x7f24880ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f248cf42640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f24880aa550 0x7f24880cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f248cf42640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f248806d1b0 con 0x7f248815bf50 2024-05-04T12:19:07.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f248cf42640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f248815ca40 con 0x7f24880aa0c0 2024-05-04T12:19:07.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f248cf42640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f248806d490 con 0x7f24880aa550 2024-05-04T12:19:07.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f248659c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f248815bf50 0x7f248815c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46830/0 (socket says 172.21.15.52:46830) 2024-05-04T12:19:07.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f248659c640 1 -- 172.21.15.52:0/2586793382 learned_addr learned my addr 172.21.15.52:0/2586793382 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:07.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f2485d9b640 1 --2- 172.21.15.52:0/2586793382 >> v2:172.21.15.52:3300/0 conn(0x7f24880aa0c0 0x7f24880ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f2486d9d640 1 --2- 172.21.15.52:0/2586793382 >> v2:172.21.15.111:3300/0 conn(0x7f24880aa550 0x7f24880cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f248559a640 1 -- 172.21.15.52:0/2586793382 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2910975145 0 0) 0x7f248806d1b0 con 0x7f248815bf50 2024-05-04T12:19:07.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f248559a640 1 -- 172.21.15.52:0/2586793382 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2470003530 con 0x7f248815bf50 2024-05-04T12:19:07.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f2485d9b640 1 -- 172.21.15.52:0/2586793382 >> v2:172.21.15.111:3300/0 conn(0x7f24880aa550 msgr2=0x7f24880cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:07.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f2485d9b640 1 --2- 172.21.15.52:0/2586793382 >> v2:172.21.15.111:3300/0 conn(0x7f24880aa550 0x7f24880cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f2485d9b640 1 -- 172.21.15.52:0/2586793382 >> v1:172.21.15.52:6789/0 conn(0x7f248815bf50 legacy=0x7f248815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.053+0000 7f2485d9b640 1 -- 172.21.15.52:0/2586793382 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248806d770 con 0x7f24880aa0c0 2024-05-04T12:19:07.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.057+0000 7f2485d9b640 1 --2- 172.21.15.52:0/2586793382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24880aa0c0 0x7f24880ccb50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f24880b2170 tx=0x7f247c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe168fd9fe25f9c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:07.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.057+0000 7f248559a640 1 -- 172.21.15.52:0/2586793382 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c0a7070 con 0x7f24880aa0c0 2024-05-04T12:19:07.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.057+0000 7f248559a640 1 -- 172.21.15.52:0/2586793382 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247c0b0070 con 0x7f24880aa0c0 2024-05-04T12:19:07.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.057+0000 7f248559a640 1 -- 172.21.15.52:0/2586793382 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c0ab870 con 0x7f24880aa0c0 2024-05-04T12:19:07.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.057+0000 7f248cf42640 1 -- 172.21.15.52:0/2586793382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24880aa0c0 msgr2=0x7f24880ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.057+0000 7f248cf42640 1 --2- 172.21.15.52:0/2586793382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24880aa0c0 0x7f24880ccb50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f24880b2170 tx=0x7f247c01c450 comp rx=0 tx=0).stop 2024-05-04T12:19:07.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.057+0000 7f248cf42640 1 -- 172.21.15.52:0/2586793382 shutdown_connections 2024-05-04T12:19:07.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.057+0000 7f248cf42640 1 --2- 172.21.15.52:0/2586793382 >> v2:172.21.15.111:3300/0 conn(0x7f24880aa550 0x7f24880cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.057+0000 7f248cf42640 1 --2- 172.21.15.52:0/2586793382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24880aa0c0 0x7f24880ccb50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.057+0000 7f248cf42640 1 -- 172.21.15.52:0/2586793382 >> 172.21.15.52:0/2586793382 conn(0x7f24880b14a0 msgr2=0x7f24880b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:07.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.057+0000 7f248cf42640 1 -- 172.21.15.52:0/2586793382 shutdown_connections 2024-05-04T12:19:07.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.057+0000 7f248cf42640 1 -- 172.21.15.52:0/2586793382 wait complete. 2024-05-04T12:19:07.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.057+0000 7f248cf42640 1 Processor -- start 2024-05-04T12:19:07.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.057+0000 7f248cf42640 1 -- start start 2024-05-04T12:19:07.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248cf42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24880aa0c0 0x7f2488147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248cf42640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24880aa550 0x7f2488152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248cf42640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f248815bf50 0x7f24881516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248cf42640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f248806ebe0 con 0x7f24880aa0c0 2024-05-04T12:19:07.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248cf42640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f248806ea60 con 0x7f24880aa550 2024-05-04T12:19:07.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248cf42640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f248806ed60 con 0x7f248815bf50 2024-05-04T12:19:07.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24880aa0c0 0x7f2488147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24880aa0c0 0x7f2488147a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56480/0 (socket says 172.21.15.52:56480) 2024-05-04T12:19:07.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248659c640 1 -- 172.21.15.52:0/1173451907 learned_addr learned my addr 172.21.15.52:0/1173451907 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:07.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248659c640 1 -- 172.21.15.52:0/1173451907 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f248815bf50 msgr2=0x7f24881516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248659c640 1 --2- 172.21.15.52:0/1173451907 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f248815bf50 0x7f24881516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248659c640 1 -- 172.21.15.52:0/1173451907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24880aa550 msgr2=0x7f2488152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f2485d9b640 1 --2- 172.21.15.52:0/1173451907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24880aa550 0x7f2488152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248659c640 1 --2- 172.21.15.52:0/1173451907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24880aa550 0x7f2488152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248659c640 1 -- 172.21.15.52:0/1173451907 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2488155300 con 0x7f24880aa0c0 2024-05-04T12:19:07.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248659c640 1 --2- 172.21.15.52:0/1173451907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24880aa0c0 0x7f2488147a10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f247800c3c0 tx=0x7f247800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:07.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f2476ffd640 1 -- 172.21.15.52:0/1173451907 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478017070 con 0x7f24880aa0c0 2024-05-04T12:19:07.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f2476ffd640 1 -- 172.21.15.52:0/1173451907 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2478005cf0 con 0x7f24880aa0c0 2024-05-04T12:19:07.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f2476ffd640 1 -- 172.21.15.52:0/1173451907 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24780072f0 con 0x7f24880aa0c0 2024-05-04T12:19:07.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248cf42640 1 -- 172.21.15.52:0/1173451907 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24881555e0 con 0x7f24880aa0c0 2024-05-04T12:19:07.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248cf42640 1 -- 172.21.15.52:0/1173451907 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2488147f50 con 0x7f24880aa0c0 2024-05-04T12:19:07.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f248cf42640 1 -- 172.21.15.52:0/1173451907 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2488062e50 con 0x7f24880aa0c0 2024-05-04T12:19:07.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f2476ffd640 1 -- 172.21.15.52:0/1173451907 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f247801a070 con 0x7f24880aa0c0 2024-05-04T12:19:07.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f2476ffd640 1 --2- 172.21.15.52:0/1173451907 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f24680678c0 0x7f2468069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f2476ffd640 1 -- 172.21.15.52:0/1173451907 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f24780853a0 con 0x7f24880aa0c0 2024-05-04T12:19:07.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.061+0000 7f2485d9b640 1 --2- 172.21.15.52:0/1173451907 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f24680678c0 0x7f2468069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.065+0000 7f2485d9b640 1 --2- 172.21.15.52:0/1173451907 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f24680678c0 0x7f2468069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f247c0af000 tx=0x7f247c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:07.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.065+0000 7f2476ffd640 1 -- 172.21.15.52:0/1173451907 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2478021020 con 0x7f24880aa0c0 2024-05-04T12:19:07.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.237+0000 7f248cf42640 1 -- 172.21.15.52:0/1173451907 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2488075320 con 0x7f24880aa0c0 2024-05-04T12:19:07.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.241+0000 7f2476ffd640 1 -- 172.21.15.52:0/1173451907 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f2478055fc0 con 0x7f24880aa0c0 2024-05-04T12:19:07.243 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-04T12:19:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.241+0000 7f248cf42640 1 -- 172.21.15.52:0/1173451907 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f24680678c0 msgr2=0x7f2468069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.241+0000 7f248cf42640 1 --2- 172.21.15.52:0/1173451907 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f24680678c0 0x7f2468069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f247c0af000 tx=0x7f247c0a7000 comp rx=0 tx=0).stop 2024-05-04T12:19:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.241+0000 7f248cf42640 1 -- 172.21.15.52:0/1173451907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24880aa0c0 msgr2=0x7f2488147a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.241+0000 7f248cf42640 1 --2- 172.21.15.52:0/1173451907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24880aa0c0 0x7f2488147a10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f247800c3c0 tx=0x7f247800c880 comp rx=0 tx=0).stop 2024-05-04T12:19:07.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.241+0000 7f248cf42640 1 -- 172.21.15.52:0/1173451907 shutdown_connections 2024-05-04T12:19:07.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.241+0000 7f248cf42640 1 --2- 172.21.15.52:0/1173451907 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f24680678c0 0x7f2468069d80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.241+0000 7f248cf42640 1 --2- 172.21.15.52:0/1173451907 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f248815bf50 0x7f24881516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.241+0000 7f248cf42640 1 --2- 172.21.15.52:0/1173451907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24880aa550 0x7f2488152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.241+0000 7f248cf42640 1 --2- 172.21.15.52:0/1173451907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24880aa0c0 0x7f2488147a10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.241+0000 7f248cf42640 1 -- 172.21.15.52:0/1173451907 >> 172.21.15.52:0/1173451907 conn(0x7f24880b14a0 msgr2=0x7f248814ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:07.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.241+0000 7f248cf42640 1 -- 172.21.15.52:0/1173451907 shutdown_connections 2024-05-04T12:19:07.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.241+0000 7f248cf42640 1 -- 172.21.15.52:0/1173451907 wait complete. 2024-05-04T12:19:07.262 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T12:19:07.262 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T12:19:07.269 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:19:07.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.385+0000 7f9313c34640 1 Processor -- start 2024-05-04T12:19:07.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.385+0000 7f9313c34640 1 -- start start 2024-05-04T12:19:07.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.385+0000 7f9313c34640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f930c0cec10 0x7f930c0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.385+0000 7f9313c34640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f930c0cb650 con 0x7f930c0ceff0 2024-05-04T12:19:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.385+0000 7f9313c34640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f930c0cb930 con 0x7f930c067070 2024-05-04T12:19:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.385+0000 7f9313c34640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f930c0cbc10 con 0x7f930c0cec10 2024-05-04T12:19:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93121d1640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f930c0ceff0 0x7f930c15e060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46836/0 (socket says 172.21.15.52:46836) 2024-05-04T12:19:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93121d1640 1 -- 172.21.15.52:0/3145623740 learned_addr learned my addr 172.21.15.52:0/3145623740 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93119d0640 1 --2- 172.21.15.52:0/3145623740 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f930c0cec10 0x7f930c0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 -- 172.21.15.52:0/3145623740 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1119921270 0 0) 0x7f930c0cb650 con 0x7f930c0ceff0 2024-05-04T12:19:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 -- 172.21.15.52:0/3145623740 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92f8003530 con 0x7f930c0ceff0 2024-05-04T12:19:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 -- 172.21.15.52:0/3145623740 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 103328487 0 0) 0x7f92f8003530 con 0x7f930c0ceff0 2024-05-04T12:19:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 -- 172.21.15.52:0/3145623740 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f930c0cec10 msgr2=0x7f930c0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 --2- 172.21.15.52:0/3145623740 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f930c0cec10 0x7f930c0ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 -- 172.21.15.52:0/3145623740 >> v1:172.21.15.111:6789/0 conn(0x7f930c067070 legacy=0x7f930c0d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 -- 172.21.15.52:0/3145623740 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c0b0570 con 0x7f930c0ceff0 2024-05-04T12:19:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 -- 172.21.15.52:0/3145623740 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f9304001a00 con 0x7f930c0ceff0 2024-05-04T12:19:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 -- 172.21.15.52:0/3145623740 >> v1:172.21.15.52:6789/0 conn(0x7f930c0ceff0 legacy=0x7f930c15e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 --2- 172.21.15.52:0/3145623740 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f930c0cec10 0x7f92f80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 --2- 172.21.15.52:0/3145623740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92f8003f70 0x7f92f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 --2- 172.21.15.52:0/3145623740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92f8004d50 0x7f92f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 -- 172.21.15.52:0/3145623740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f930c0cbc10 con 0x7f92f8004d50 2024-05-04T12:19:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 -- 172.21.15.52:0/3145623740 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f92f800bab0 con 0x7f92f8003f70 2024-05-04T12:19:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 -- 172.21.15.52:0/3145623740 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f92f800bcb0 con 0x7f930c0cec10 2024-05-04T12:19:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93121d1640 1 --2- 172.21.15.52:0/3145623740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92f8004d50 0x7f92f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93111cf640 1 --2- 172.21.15.52:0/3145623740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92f8003f70 0x7f92f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93119d0640 1 --2- 172.21.15.52:0/3145623740 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f930c0cec10 0x7f92f80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93121d1640 1 -- 172.21.15.52:0/3145623740 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f930c0cec10 msgr2=0x7f92f80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93121d1640 1 --2- 172.21.15.52:0/3145623740 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f930c0cec10 0x7f92f80039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93121d1640 1 -- 172.21.15.52:0/3145623740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92f8003f70 msgr2=0x7f92f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93121d1640 1 --2- 172.21.15.52:0/3145623740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92f8003f70 0x7f92f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93121d1640 1 -- 172.21.15.52:0/3145623740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f8003530 con 0x7f92f8004d50 2024-05-04T12:19:07.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93121d1640 1 --2- 172.21.15.52:0/3145623740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92f8004d50 0x7f92f80051d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f930400a840 tx=0x7f930401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3a2d922e26f8b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:07.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 -- 172.21.15.52:0/3145623740 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930400a460 con 0x7f92f8004d50 2024-05-04T12:19:07.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 -- 172.21.15.52:0/3145623740 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93040ad630 con 0x7f92f8004d50 2024-05-04T12:19:07.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.389+0000 7f93109ce640 1 -- 172.21.15.52:0/3145623740 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93040ada60 con 0x7f92f8004d50 2024-05-04T12:19:07.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 -- 172.21.15.52:0/3145623740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92f8004d50 msgr2=0x7f92f80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 --2- 172.21.15.52:0/3145623740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92f8004d50 0x7f92f80051d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f930400a840 tx=0x7f930401d360 comp rx=0 tx=0).stop 2024-05-04T12:19:07.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93121d1640 1 -- 172.21.15.52:0/3145623740 reap_dead start 2024-05-04T12:19:07.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 -- 172.21.15.52:0/3145623740 shutdown_connections 2024-05-04T12:19:07.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 -- 172.21.15.52:0/3145623740 >> 172.21.15.52:0/3145623740 conn(0x7f930c0cd5b0 msgr2=0x7f930c0cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:07.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 -- 172.21.15.52:0/3145623740 shutdown_connections 2024-05-04T12:19:07.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 -- 172.21.15.52:0/3145623740 wait complete. 2024-05-04T12:19:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 Processor -- start 2024-05-04T12:19:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 -- start start 2024-05-04T12:19:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f930c06e1a0 0x7f930c072c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f930c070600 0x7f930c06e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f930c1f7a50 0x7f930c1f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f930c0adc20 con 0x7f930c070600 2024-05-04T12:19:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f930c0adaa0 con 0x7f930c06e1a0 2024-05-04T12:19:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f930c15e770 con 0x7f930c1f7a50 2024-05-04T12:19:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93111cf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f930c070600 0x7f930c06e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93111cf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f930c070600 0x7f930c06e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56504/0 (socket says 172.21.15.52:56504) 2024-05-04T12:19:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93111cf640 1 -- 172.21.15.52:0/3172607044 learned_addr learned my addr 172.21.15.52:0/3172607044 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93119d0640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f930c06e1a0 0x7f930c072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93121d1640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f930c1f7a50 0x7f930c1f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93111cf640 1 -- 172.21.15.52:0/3172607044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f930c1f7a50 msgr2=0x7f930c1f7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93111cf640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f930c1f7a50 0x7f930c1f7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93111cf640 1 -- 172.21.15.52:0/3172607044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f930c06e1a0 msgr2=0x7f930c072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93111cf640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f930c06e1a0 0x7f930c072c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93111cf640 1 -- 172.21.15.52:0/3172607044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c06eac0 con 0x7f930c070600 2024-05-04T12:19:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93121d1640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f930c1f7a50 0x7f930c1f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93111cf640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f930c070600 0x7f930c06e580 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00e510 tx=0x7f92fc00e9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93027fc640 1 -- 172.21.15.52:0/3172607044 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc0052e0 con 0x7f930c070600 2024-05-04T12:19:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93027fc640 1 -- 172.21.15.52:0/3172607044 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92fc018070 con 0x7f930c070600 2024-05-04T12:19:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 -- 172.21.15.52:0/3172607044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930c070de0 con 0x7f930c070600 2024-05-04T12:19:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f93027fc640 1 -- 172.21.15.52:0/3172607044 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc003820 con 0x7f930c070600 2024-05-04T12:19:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 -- 172.21.15.52:0/3172607044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f930c0713a0 con 0x7f930c070600 2024-05-04T12:19:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.393+0000 7f9313c34640 1 -- 172.21.15.52:0/3172607044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f930c062e50 con 0x7f930c070600 2024-05-04T12:19:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.397+0000 7f93027fc640 1 -- 172.21.15.52:0/3172607044 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f92fc0039c0 con 0x7f930c070600 2024-05-04T12:19:07.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.397+0000 7f93027fc640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f92f00678c0 0x7f92f0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.397+0000 7f93119d0640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f92f00678c0 0x7f92f0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.397+0000 7f93027fc640 1 -- 172.21.15.52:0/3172607044 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f92fc085950 con 0x7f930c070600 2024-05-04T12:19:07.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.397+0000 7f93119d0640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f92f00678c0 0x7f92f0069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9308009a30 tx=0x7f9308008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.401+0000 7f93027fc640 1 -- 172.21.15.52:0/3172607044 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f92fc056400 con 0x7f930c070600 2024-05-04T12:19:07.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.553+0000 7f9313c34640 1 -- 172.21.15.52:0/3172607044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f930c0c7fb0 con 0x7f930c070600 2024-05-04T12:19:07.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.557+0000 7f93027fc640 1 -- 172.21.15.52:0/3172607044 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16913 (secure 0 0 0) 0x7f92fc005480 con 0x7f930c070600 2024-05-04T12:19:07.560 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:07.560 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:19:04.046290+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:19:07.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.557+0000 7f9313c34640 1 -- 172.21.15.52:0/3172607044 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f92f00678c0 msgr2=0x7f92f0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.557+0000 7f9313c34640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f92f00678c0 0x7f92f0069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9308009a30 tx=0x7f9308008040 comp rx=0 tx=0).stop 2024-05-04T12:19:07.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.557+0000 7f9313c34640 1 -- 172.21.15.52:0/3172607044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f930c070600 msgr2=0x7f930c06e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.557+0000 7f9313c34640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f930c070600 0x7f930c06e580 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00e510 tx=0x7f92fc00e9d0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.557+0000 7f9313c34640 1 -- 172.21.15.52:0/3172607044 shutdown_connections 2024-05-04T12:19:07.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.557+0000 7f9313c34640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f92f00678c0 0x7f92f0069d80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.557+0000 7f9313c34640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f930c1f7a50 0x7f930c1f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.557+0000 7f9313c34640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f930c070600 0x7f930c06e580 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.557+0000 7f9313c34640 1 --2- 172.21.15.52:0/3172607044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f930c06e1a0 0x7f930c072c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.557+0000 7f9313c34640 1 -- 172.21.15.52:0/3172607044 >> 172.21.15.52:0/3172607044 conn(0x7f930c0cd5b0 msgr2=0x7f930c0c76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:07.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.557+0000 7f9313c34640 1 -- 172.21.15.52:0/3172607044 shutdown_connections 2024-05-04T12:19:07.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.557+0000 7f9313c34640 1 -- 172.21.15.52:0/3172607044 wait complete. 2024-05-04T12:19:07.577 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:19:07.578 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:19:07.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.697+0000 7f5a2482e640 1 Processor -- start 2024-05-04T12:19:07.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.697+0000 7f5a2482e640 1 -- start start 2024-05-04T12:19:07.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a2482e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a2015bd90 0x7f5a200cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a2482e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5a2006fa80 0x7f5a200cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a2482e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a200ccb80 con 0x7f5a2015c170 2024-05-04T12:19:07.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a2482e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f5a200cc930 con 0x7f5a2006fa80 2024-05-04T12:19:07.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a2482e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5a200cd4b0 con 0x7f5a2015bd90 2024-05-04T12:19:07.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a1e59c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a2015bd90 0x7f5a200cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a1ed9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5a2006fa80 0x7f5a200cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a1ed9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5a2006fa80 0x7f5a200cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:34458/0 (socket says 172.21.15.52:34458) 2024-05-04T12:19:07.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a1dd9b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5a2015c170 0x7f5a2006f370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.52:34760/0 (socket says 172.21.15.52:34760) 2024-05-04T12:19:07.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a1e59c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a2015bd90 0x7f5a200cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.52:55178/0 (socket says 172.21.15.52:55178) 2024-05-04T12:19:07.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a1ed9d640 1 -- 172.21.15.52:0/209052832 learned_addr learned my addr 172.21.15.52:0/209052832 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:07.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a1e59c640 1 -- 172.21.15.52:0/209052832 >> v2:172.21.15.111:3300/0 conn(0x7f5a2006fa80 msgr2=0x7f5a200cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:07.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a1e59c640 1 --2- 172.21.15.52:0/209052832 >> v2:172.21.15.111:3300/0 conn(0x7f5a2006fa80 0x7f5a200cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a1e59c640 1 -- 172.21.15.52:0/209052832 >> v1:172.21.15.111:6789/0 conn(0x7f5a2015c170 legacy=0x7f5a2006f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a1e59c640 1 -- 172.21.15.52:0/209052832 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a200cce60 con 0x7f5a2015bd90 2024-05-04T12:19:07.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.701+0000 7f5a1ed9d640 1 --2- 172.21.15.52:0/209052832 >> v2:172.21.15.111:3300/0 conn(0x7f5a2006fa80 0x7f5a200cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:19:07.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.705+0000 7f5a1e59c640 1 --2- 172.21.15.52:0/209052832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a2015bd90 0x7f5a200cf060 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5a100097c0 tx=0x7f5a1001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfa462e42422a762 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:07.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.705+0000 7f5a1d59a640 1 -- 172.21.15.52:0/209052832 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a100b0070 con 0x7f5a2015bd90 2024-05-04T12:19:07.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.705+0000 7f5a1d59a640 1 -- 172.21.15.52:0/209052832 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a100ab600 con 0x7f5a2015bd90 2024-05-04T12:19:07.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.705+0000 7f5a1d59a640 1 -- 172.21.15.52:0/209052832 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a100ab9d0 con 0x7f5a2015bd90 2024-05-04T12:19:07.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.705+0000 7f5a2482e640 1 -- 172.21.15.52:0/209052832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a2015bd90 msgr2=0x7f5a200cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.705+0000 7f5a2482e640 1 --2- 172.21.15.52:0/209052832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a2015bd90 0x7f5a200cf060 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5a100097c0 tx=0x7f5a1001c410 comp rx=0 tx=0).stop 2024-05-04T12:19:07.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.705+0000 7f5a2482e640 1 -- 172.21.15.52:0/209052832 shutdown_connections 2024-05-04T12:19:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.705+0000 7f5a2482e640 1 --2- 172.21.15.52:0/209052832 >> v2:172.21.15.111:3300/0 conn(0x7f5a2006fa80 0x7f5a200cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.705+0000 7f5a2482e640 1 --2- 172.21.15.52:0/209052832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a2015bd90 0x7f5a200cf060 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.705+0000 7f5a2482e640 1 -- 172.21.15.52:0/209052832 >> 172.21.15.52:0/209052832 conn(0x7f5a20067730 msgr2=0x7f5a200b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.705+0000 7f5a2482e640 1 -- 172.21.15.52:0/209052832 shutdown_connections 2024-05-04T12:19:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.705+0000 7f5a2482e640 1 -- 172.21.15.52:0/209052832 wait complete. 2024-05-04T12:19:07.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.705+0000 7f5a2482e640 1 Processor -- start 2024-05-04T12:19:07.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.705+0000 7f5a2482e640 1 -- start start 2024-05-04T12:19:07.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a2482e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a2006fa80 0x7f5a201fba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a2482e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a2015bd90 0x7f5a201525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a2482e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a2015c170 0x7f5a201f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a2482e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2006d7e0 con 0x7f5a2006fa80 2024-05-04T12:19:07.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a2482e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2006d660 con 0x7f5a2015c170 2024-05-04T12:19:07.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a2482e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5a2006d960 con 0x7f5a2015bd90 2024-05-04T12:19:07.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a1e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a2006fa80 0x7f5a201fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a1e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a2006fa80 0x7f5a201fba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56506/0 (socket says 172.21.15.52:56506) 2024-05-04T12:19:07.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a1e59c640 1 -- 172.21.15.52:0/1934173334 learned_addr learned my addr 172.21.15.52:0/1934173334 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:07.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a1ed9d640 1 --2- 172.21.15.52:0/1934173334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a2015c170 0x7f5a201f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a1dd9b640 1 --2- 172.21.15.52:0/1934173334 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a2015bd90 0x7f5a201525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a1e59c640 1 -- 172.21.15.52:0/1934173334 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a2015bd90 msgr2=0x7f5a201525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a1e59c640 1 --2- 172.21.15.52:0/1934173334 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a2015bd90 0x7f5a201525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a1e59c640 1 -- 172.21.15.52:0/1934173334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a2015c170 msgr2=0x7f5a201f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a1e59c640 1 --2- 172.21.15.52:0/1934173334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a2015c170 0x7f5a201f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a1e59c640 1 -- 172.21.15.52:0/1934173334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a20152d30 con 0x7f5a2006fa80 2024-05-04T12:19:07.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a1e59c640 1 --2- 172.21.15.52:0/1934173334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a2006fa80 0x7f5a201fba10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5a100ae040 tx=0x7f5a1001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:07.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a0effd640 1 -- 172.21.15.52:0/1934173334 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a100b0050 con 0x7f5a2006fa80 2024-05-04T12:19:07.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a0effd640 1 -- 172.21.15.52:0/1934173334 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a100bc070 con 0x7f5a2006fa80 2024-05-04T12:19:07.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a2482e640 1 -- 172.21.15.52:0/1934173334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a20153010 con 0x7f5a2006fa80 2024-05-04T12:19:07.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a0effd640 1 -- 172.21.15.52:0/1934173334 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10006e20 con 0x7f5a2006fa80 2024-05-04T12:19:07.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a2482e640 1 -- 172.21.15.52:0/1934173334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a201535d0 con 0x7f5a2006fa80 2024-05-04T12:19:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a2482e640 1 -- 172.21.15.52:0/1934173334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a20153810 con 0x7f5a2006fa80 2024-05-04T12:19:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a0effd640 1 -- 172.21.15.52:0/1934173334 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5a100b5870 con 0x7f5a2006fa80 2024-05-04T12:19:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a0effd640 1 --2- 172.21.15.52:0/1934173334 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5a000678c0 0x7f5a00069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a0effd640 1 -- 172.21.15.52:0/1934173334 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f5a10120390 con 0x7f5a2006fa80 2024-05-04T12:19:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a1dd9b640 1 --2- 172.21.15.52:0/1934173334 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5a000678c0 0x7f5a00069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.709+0000 7f5a1dd9b640 1 --2- 172.21.15.52:0/1934173334 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5a000678c0 0x7f5a00069d80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5a2014b760 tx=0x7f5a14007cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.713+0000 7f5a0effd640 1 -- 172.21.15.52:0/1934173334 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5a20153810 con 0x7f5a2006fa80 2024-05-04T12:19:07.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.873+0000 7f5a2482e640 1 -- 172.21.15.52:0/1934173334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5a20062e50 con 0x7f5a2006fa80 2024-05-04T12:19:07.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.873+0000 7f5a0effd640 1 -- 172.21.15.52:0/1934173334 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f5a10016090 con 0x7f5a2006fa80 2024-05-04T12:19:07.877 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:19:07.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.873+0000 7f5a2482e640 1 -- 172.21.15.52:0/1934173334 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5a000678c0 msgr2=0x7f5a00069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.873+0000 7f5a2482e640 1 --2- 172.21.15.52:0/1934173334 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5a000678c0 0x7f5a00069d80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5a2014b760 tx=0x7f5a14007cd0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.873+0000 7f5a2482e640 1 -- 172.21.15.52:0/1934173334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a2006fa80 msgr2=0x7f5a201fba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:07.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.873+0000 7f5a2482e640 1 --2- 172.21.15.52:0/1934173334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a2006fa80 0x7f5a201fba10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5a100ae040 tx=0x7f5a1001ac50 comp rx=0 tx=0).stop 2024-05-04T12:19:07.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.873+0000 7f5a2482e640 1 -- 172.21.15.52:0/1934173334 shutdown_connections 2024-05-04T12:19:07.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.873+0000 7f5a2482e640 1 --2- 172.21.15.52:0/1934173334 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5a000678c0 0x7f5a00069d80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.873+0000 7f5a2482e640 1 --2- 172.21.15.52:0/1934173334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a2015c170 0x7f5a201f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.873+0000 7f5a2482e640 1 --2- 172.21.15.52:0/1934173334 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a2015bd90 0x7f5a201525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.873+0000 7f5a2482e640 1 --2- 172.21.15.52:0/1934173334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a2006fa80 0x7f5a201fba10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:07.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.873+0000 7f5a2482e640 1 -- 172.21.15.52:0/1934173334 >> 172.21.15.52:0/1934173334 conn(0x7f5a20067730 msgr2=0x7f5a2014e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:07.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.873+0000 7f5a2482e640 1 -- 172.21.15.52:0/1934173334 shutdown_connections 2024-05-04T12:19:07.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:07.873+0000 7f5a2482e640 1 -- 172.21.15.52:0/1934173334 wait complete. 2024-05-04T12:19:07.894 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f016166b640 1 Processor -- start 2024-05-04T12:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f016166b640 1 -- start start 2024-05-04T12:19:08.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f016166b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f015c141210 0x7f015c162970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f016166b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f015c070690 0x7f015c070b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f016166b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f015c15f450 con 0x7f015c15e7b0 2024-05-04T12:19:08.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f016166b640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f015c0aee10 con 0x7f015c070690 2024-05-04T12:19:08.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f016166b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f015c1415f0 con 0x7f015c141210 2024-05-04T12:19:08.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f015a7fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f015c070690 0x7f015c070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f015a7fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f015c070690 0x7f015c070b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56514/0 (socket says 172.21.15.52:56514) 2024-05-04T12:19:08.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f015a7fc640 1 -- 172.21.15.52:0/1170263834 learned_addr learned my addr 172.21.15.52:0/1170263834 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:08.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f015affd640 1 --2- 172.21.15.52:0/1170263834 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f015c141210 0x7f015c162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f015a7fc640 1 -- 172.21.15.52:0/1170263834 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f015c141210 msgr2=0x7f015c162970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:08.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f015a7fc640 1 --2- 172.21.15.52:0/1170263834 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f015c141210 0x7f015c162970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f015a7fc640 1 -- 172.21.15.52:0/1170263834 >> v1:172.21.15.111:6789/0 conn(0x7f015c15e7b0 legacy=0x7f015c15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.021+0000 7f015a7fc640 1 -- 172.21.15.52:0/1170263834 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c15f730 con 0x7f015c070690 2024-05-04T12:19:08.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.025+0000 7f015a7fc640 1 --2- 172.21.15.52:0/1170263834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c070690 0x7f015c070b10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f01500098a0 tx=0x7f015001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ba9b9ef9950b9b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:08.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.025+0000 7f0159ffb640 1 -- 172.21.15.52:0/1170263834 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01500b1070 con 0x7f015c070690 2024-05-04T12:19:08.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.025+0000 7f0159ffb640 1 -- 172.21.15.52:0/1170263834 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01500ac600 con 0x7f015c070690 2024-05-04T12:19:08.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.025+0000 7f0159ffb640 1 -- 172.21.15.52:0/1170263834 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01500ac9d0 con 0x7f015c070690 2024-05-04T12:19:08.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.025+0000 7f016166b640 1 -- 172.21.15.52:0/1170263834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c070690 msgr2=0x7f015c070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.025+0000 7f016166b640 1 --2- 172.21.15.52:0/1170263834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c070690 0x7f015c070b10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f01500098a0 tx=0x7f015001c4b0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.025+0000 7f016166b640 1 -- 172.21.15.52:0/1170263834 shutdown_connections 2024-05-04T12:19:08.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.025+0000 7f016166b640 1 --2- 172.21.15.52:0/1170263834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c070690 0x7f015c070b10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.025+0000 7f016166b640 1 --2- 172.21.15.52:0/1170263834 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f015c141210 0x7f015c162970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.025+0000 7f016166b640 1 -- 172.21.15.52:0/1170263834 >> 172.21.15.52:0/1170263834 conn(0x7f015c067730 msgr2=0x7f015c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:08.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.025+0000 7f016166b640 1 -- 172.21.15.52:0/1170263834 shutdown_connections 2024-05-04T12:19:08.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.025+0000 7f016166b640 1 -- 172.21.15.52:0/1170263834 wait complete. 2024-05-04T12:19:08.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f016166b640 1 Processor -- start 2024-05-04T12:19:08.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f016166b640 1 -- start start 2024-05-04T12:19:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f016166b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c070690 0x7f015c1384e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f016166b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f015c141210 0x7f015c13ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f016166b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c15e7b0 0x7f015c13fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f016166b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f015c0cc790 con 0x7f015c070690 2024-05-04T12:19:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f016166b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f015c0cc610 con 0x7f015c15e7b0 2024-05-04T12:19:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f016166b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f015c0cc910 con 0x7f015c141210 2024-05-04T12:19:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f015affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c070690 0x7f015c1384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f015affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c070690 0x7f015c1384e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56522/0 (socket says 172.21.15.52:56522) 2024-05-04T12:19:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f015affd640 1 -- 172.21.15.52:0/1737599782 learned_addr learned my addr 172.21.15.52:0/1737599782 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f015b7fe640 1 --2- 172.21.15.52:0/1737599782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c15e7b0 0x7f015c13fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f015a7fc640 1 --2- 172.21.15.52:0/1737599782 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f015c141210 0x7f015c13ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f015affd640 1 -- 172.21.15.52:0/1737599782 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f015c141210 msgr2=0x7f015c13ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f015affd640 1 --2- 172.21.15.52:0/1737599782 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f015c141210 0x7f015c13ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f015affd640 1 -- 172.21.15.52:0/1737599782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c15e7b0 msgr2=0x7f015c13fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f015affd640 1 --2- 172.21.15.52:0/1737599782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c15e7b0 0x7f015c13fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f015affd640 1 -- 172.21.15.52:0/1737599782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c13e440 con 0x7f015c070690 2024-05-04T12:19:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f015affd640 1 --2- 172.21.15.52:0/1737599782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c070690 0x7f015c1384e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f014c00d5f0 tx=0x7f014c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f01437fe640 1 -- 172.21.15.52:0/1737599782 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c0125f0 con 0x7f015c070690 2024-05-04T12:19:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f01437fe640 1 -- 172.21.15.52:0/1737599782 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f014c019070 con 0x7f015c070690 2024-05-04T12:19:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f01437fe640 1 -- 172.21.15.52:0/1737599782 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c0129b0 con 0x7f015c070690 2024-05-04T12:19:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f016166b640 1 -- 172.21.15.52:0/1737599782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015c13ca00 con 0x7f015c070690 2024-05-04T12:19:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f016166b640 1 -- 172.21.15.52:0/1737599782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015c13c720 con 0x7f015c070690 2024-05-04T12:19:08.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f016166b640 1 -- 172.21.15.52:0/1737599782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015c062e50 con 0x7f015c070690 2024-05-04T12:19:08.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f01437fe640 1 -- 172.21.15.52:0/1737599782 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f014c012b50 con 0x7f015c070690 2024-05-04T12:19:08.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.029+0000 7f01437fe640 1 --2- 172.21.15.52:0/1737599782 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f01480678c0 0x7f0148069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.033+0000 7f01437fe640 1 -- 172.21.15.52:0/1737599782 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f014c0860f0 con 0x7f015c070690 2024-05-04T12:19:08.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.033+0000 7f015a7fc640 1 --2- 172.21.15.52:0/1737599782 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f01480678c0 0x7f0148069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.033+0000 7f015a7fc640 1 --2- 172.21.15.52:0/1737599782 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f01480678c0 0x7f0148069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f01500ae000 tx=0x7f0150008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:08.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.037+0000 7f01437fe640 1 -- 172.21.15.52:0/1737599782 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f014c011340 con 0x7f015c070690 2024-05-04T12:19:08.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.197+0000 7f016166b640 1 -- 172.21.15.52:0/1737599782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f015c14f380 con 0x7f015c070690 2024-05-04T12:19:08.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.197+0000 7f01437fe640 1 -- 172.21.15.52:0/1737599782 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f014c056ba0 con 0x7f015c070690 2024-05-04T12:19:08.202 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:19:08.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.201+0000 7f016166b640 1 -- 172.21.15.52:0/1737599782 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f01480678c0 msgr2=0x7f0148069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.201+0000 7f016166b640 1 --2- 172.21.15.52:0/1737599782 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f01480678c0 0x7f0148069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f01500ae000 tx=0x7f0150008040 comp rx=0 tx=0).stop 2024-05-04T12:19:08.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.201+0000 7f016166b640 1 -- 172.21.15.52:0/1737599782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c070690 msgr2=0x7f015c1384e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.201+0000 7f016166b640 1 --2- 172.21.15.52:0/1737599782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c070690 0x7f015c1384e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f014c00d5f0 tx=0x7f014c00dab0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.201+0000 7f016166b640 1 -- 172.21.15.52:0/1737599782 shutdown_connections 2024-05-04T12:19:08.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.201+0000 7f016166b640 1 --2- 172.21.15.52:0/1737599782 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f01480678c0 0x7f0148069d80 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f01500ae000 tx=0x7f0150008040 comp rx=0 tx=0).stop 2024-05-04T12:19:08.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.201+0000 7f016166b640 1 --2- 172.21.15.52:0/1737599782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c15e7b0 0x7f015c13fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.201+0000 7f016166b640 1 --2- 172.21.15.52:0/1737599782 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f015c141210 0x7f015c13ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.201+0000 7f016166b640 1 --2- 172.21.15.52:0/1737599782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c070690 0x7f015c1384e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.201+0000 7f016166b640 1 -- 172.21.15.52:0/1737599782 >> 172.21.15.52:0/1737599782 conn(0x7f015c067730 msgr2=0x7f015c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:08.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.201+0000 7f016166b640 1 -- 172.21.15.52:0/1737599782 shutdown_connections 2024-05-04T12:19:08.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.201+0000 7f016166b640 1 -- 172.21.15.52:0/1737599782 wait complete. 2024-05-04T12:19:08.220 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:19:08.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.333+0000 7f43356f4640 1 Processor -- start 2024-05-04T12:19:08.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f43356f4640 1 -- start start 2024-05-04T12:19:08.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f43356f4640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4330145960 0x7f4330160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f43356f4640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433006dbf0 con 0x7f433015c590 2024-05-04T12:19:08.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f43356f4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4330160ad0 con 0x7f43300ccea0 2024-05-04T12:19:08.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f43356f4640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f433006cfe0 con 0x7f4330145960 2024-05-04T12:19:08.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432e7fc640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f433015c590 0x7f43300cc790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46840/0 (socket says 172.21.15.52:46840) 2024-05-04T12:19:08.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432e7fc640 1 -- 172.21.15.52:0/1899269041 learned_addr learned my addr 172.21.15.52:0/1899269041 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432effd640 1 --2- 172.21.15.52:0/1899269041 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4330145960 0x7f4330160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 -- 172.21.15.52:0/1899269041 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3081407163 0 0) 0x7f433006dbf0 con 0x7f433015c590 2024-05-04T12:19:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 -- 172.21.15.52:0/1899269041 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4318003530 con 0x7f433015c590 2024-05-04T12:19:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 -- 172.21.15.52:0/1899269041 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1634405309 0 0) 0x7f4318003530 con 0x7f433015c590 2024-05-04T12:19:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 -- 172.21.15.52:0/1899269041 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4330145960 msgr2=0x7f4330160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 --2- 172.21.15.52:0/1899269041 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4330145960 0x7f4330160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 -- 172.21.15.52:0/1899269041 >> v1:172.21.15.111:6789/0 conn(0x7f43300ccea0 legacy=0x7f433006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 -- 172.21.15.52:0/1899269041 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433006d2c0 con 0x7f433015c590 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 -- 172.21.15.52:0/1899269041 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f4324001680 con 0x7f433015c590 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 -- 172.21.15.52:0/1899269041 >> v1:172.21.15.52:6789/0 conn(0x7f433015c590 legacy=0x7f43300cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 --2- 172.21.15.52:0/1899269041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43180039f0 0x7f4318003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 --2- 172.21.15.52:0/1899269041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43180043a0 0x7f4318004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 --2- 172.21.15.52:0/1899269041 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4330145960 0x7f4318005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 -- 172.21.15.52:0/1899269041 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f433006cfe0 con 0x7f43180039f0 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 -- 172.21.15.52:0/1899269041 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f431800ba80 con 0x7f43180043a0 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432dffb640 1 -- 172.21.15.52:0/1899269041 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f431800bc80 con 0x7f4330145960 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432effd640 1 --2- 172.21.15.52:0/1899269041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43180039f0 0x7f4318003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432e7fc640 1 --2- 172.21.15.52:0/1899269041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43180043a0 0x7f4318004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432f7fe640 1 --2- 172.21.15.52:0/1899269041 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4330145960 0x7f4318005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432effd640 1 -- 172.21.15.52:0/1899269041 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4330145960 msgr2=0x7f4318005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432effd640 1 --2- 172.21.15.52:0/1899269041 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4330145960 0x7f4318005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432effd640 1 -- 172.21.15.52:0/1899269041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43180043a0 msgr2=0x7f4318004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432effd640 1 --2- 172.21.15.52:0/1899269041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43180043a0 0x7f4318004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432effd640 1 -- 172.21.15.52:0/1899269041 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4318003530 con 0x7f43180039f0 2024-05-04T12:19:08.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.337+0000 7f432f7fe640 1 --2- 172.21.15.52:0/1899269041 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4330145960 0x7f4318005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f432effd640 1 --2- 172.21.15.52:0/1899269041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43180039f0 0x7f4318003dd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4320009ac0 tx=0x7f432001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a290f26729f37ab7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f432dffb640 1 -- 172.21.15.52:0/1899269041 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320009690 con 0x7f43180039f0 2024-05-04T12:19:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f432dffb640 1 -- 172.21.15.52:0/1899269041 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43200a6510 con 0x7f43180039f0 2024-05-04T12:19:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f432dffb640 1 -- 172.21.15.52:0/1899269041 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43200a68e0 con 0x7f43180039f0 2024-05-04T12:19:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f43356f4640 1 -- 172.21.15.52:0/1899269041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43180039f0 msgr2=0x7f4318003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f43356f4640 1 --2- 172.21.15.52:0/1899269041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43180039f0 0x7f4318003dd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4320009ac0 tx=0x7f432001ca20 comp rx=0 tx=0).stop 2024-05-04T12:19:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f432f7fe640 1 -- 172.21.15.52:0/1899269041 reap_dead start 2024-05-04T12:19:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f43356f4640 1 -- 172.21.15.52:0/1899269041 shutdown_connections 2024-05-04T12:19:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f43356f4640 1 -- 172.21.15.52:0/1899269041 >> 172.21.15.52:0/1899269041 conn(0x7f4330067730 msgr2=0x7f43300b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f43356f4640 1 -- 172.21.15.52:0/1899269041 shutdown_connections 2024-05-04T12:19:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f43356f4640 1 -- 172.21.15.52:0/1899269041 wait complete. 2024-05-04T12:19:08.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f43356f4640 1 Processor -- start 2024-05-04T12:19:08.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f43356f4640 1 -- start start 2024-05-04T12:19:08.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f43356f4640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f43180039f0 0x7f43301f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f43356f4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43301376c0 0x7f433013ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f43356f4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4330143eb0 0x7f4330144330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f43356f4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f433006cdc0 con 0x7f4330143eb0 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f43356f4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f433006cc40 con 0x7f43301376c0 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f43356f4640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4330145d40 con 0x7f43180039f0 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f432f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4330143eb0 0x7f4330144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f432f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4330143eb0 0x7f4330144330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56532/0 (socket says 172.21.15.52:56532) 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f432f7fe640 1 -- 172.21.15.52:0/2847084311 learned_addr learned my addr 172.21.15.52:0/2847084311 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f432e7fc640 1 --2- 172.21.15.52:0/2847084311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43301376c0 0x7f433013ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.341+0000 7f432effd640 1 --2- 172.21.15.52:0/2847084311 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f43180039f0 0x7f43301f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f432f7fe640 1 -- 172.21.15.52:0/2847084311 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f43180039f0 msgr2=0x7f43301f9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f432f7fe640 1 --2- 172.21.15.52:0/2847084311 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f43180039f0 0x7f43301f9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f432f7fe640 1 -- 172.21.15.52:0/2847084311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43301376c0 msgr2=0x7f433013ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f432f7fe640 1 --2- 172.21.15.52:0/2847084311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43301376c0 0x7f433013ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f432f7fe640 1 -- 172.21.15.52:0/2847084311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433013b190 con 0x7f4330143eb0 2024-05-04T12:19:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f432f7fe640 1 --2- 172.21.15.52:0/2847084311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4330143eb0 0x7f4330144330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f43300af2b0 tx=0x7f432800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:08.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f43177fe640 1 -- 172.21.15.52:0/2847084311 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328018070 con 0x7f4330143eb0 2024-05-04T12:19:08.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f43177fe640 1 -- 172.21.15.52:0/2847084311 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4328003ba0 con 0x7f4330143eb0 2024-05-04T12:19:08.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f43356f4640 1 -- 172.21.15.52:0/2847084311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4330137b40 con 0x7f4330143eb0 2024-05-04T12:19:08.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f43177fe640 1 -- 172.21.15.52:0/2847084311 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328005070 con 0x7f4330143eb0 2024-05-04T12:19:08.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f43356f4640 1 -- 172.21.15.52:0/2847084311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433013b770 con 0x7f4330143eb0 2024-05-04T12:19:08.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f43356f4640 1 -- 172.21.15.52:0/2847084311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4330062e50 con 0x7f4330143eb0 2024-05-04T12:19:08.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f43177fe640 1 -- 172.21.15.52:0/2847084311 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4328005210 con 0x7f4330143eb0 2024-05-04T12:19:08.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f43177fe640 1 --2- 172.21.15.52:0/2847084311 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f431c0678f0 0x7f431c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f43177fe640 1 -- 172.21.15.52:0/2847084311 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f4328085270 con 0x7f4330143eb0 2024-05-04T12:19:08.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f432effd640 1 --2- 172.21.15.52:0/2847084311 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f431c0678f0 0x7f431c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.345+0000 7f432effd640 1 --2- 172.21.15.52:0/2847084311 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f431c0678f0 0x7f431c069db0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f43200af000 tx=0x7f43200046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:08.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.349+0000 7f43177fe640 1 -- 172.21.15.52:0/2847084311 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f433013b190 con 0x7f4330143eb0 2024-05-04T12:19:08.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.509+0000 7f43356f4640 1 -- 172.21.15.52:0/2847084311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4330160370 con 0x7f4330143eb0 2024-05-04T12:19:08.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.509+0000 7f43177fe640 1 -- 172.21.15.52:0/2847084311 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f4328055d20 con 0x7f4330143eb0 2024-05-04T12:19:08.515 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:19:08.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.513+0000 7f43356f4640 1 -- 172.21.15.52:0/2847084311 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f431c0678f0 msgr2=0x7f431c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.513+0000 7f43356f4640 1 --2- 172.21.15.52:0/2847084311 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f431c0678f0 0x7f431c069db0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f43200af000 tx=0x7f43200046d0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.513+0000 7f43356f4640 1 -- 172.21.15.52:0/2847084311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4330143eb0 msgr2=0x7f4330144330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.513+0000 7f43356f4640 1 --2- 172.21.15.52:0/2847084311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4330143eb0 0x7f4330144330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f43300af2b0 tx=0x7f432800dbe0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.513+0000 7f43356f4640 1 -- 172.21.15.52:0/2847084311 shutdown_connections 2024-05-04T12:19:08.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.513+0000 7f43356f4640 1 --2- 172.21.15.52:0/2847084311 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f431c0678f0 0x7f431c069db0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.513+0000 7f43356f4640 1 --2- 172.21.15.52:0/2847084311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4330143eb0 0x7f4330144330 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.513+0000 7f43356f4640 1 --2- 172.21.15.52:0/2847084311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43301376c0 0x7f433013ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.513+0000 7f43356f4640 1 --2- 172.21.15.52:0/2847084311 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f43180039f0 0x7f43301f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.513+0000 7f43356f4640 1 -- 172.21.15.52:0/2847084311 >> 172.21.15.52:0/2847084311 conn(0x7f4330067730 msgr2=0x7f43300b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:08.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.513+0000 7f43356f4640 1 -- 172.21.15.52:0/2847084311 shutdown_connections 2024-05-04T12:19:08.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.513+0000 7f43356f4640 1 -- 172.21.15.52:0/2847084311 wait complete. 2024-05-04T12:19:08.534 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-04T12:19:08.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14ef547640 1 Processor -- start 2024-05-04T12:19:08.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14ef547640 1 -- start start 2024-05-04T12:19:08.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14ef547640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f14e815bf50 0x7f14e815c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14ef547640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f14e80cd260 0x7f14e806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14ef547640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14e806d1b0 con 0x7f14e80aa0c0 2024-05-04T12:19:08.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14ef547640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f14e806d490 con 0x7f14e80cd260 2024-05-04T12:19:08.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14ef547640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f14e806d770 con 0x7f14e815bf50 2024-05-04T12:19:08.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14edae4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f14e80cd260 0x7f14e806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14edae4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f14e80cd260 0x7f14e806e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56534/0 (socket says 172.21.15.52:56534) 2024-05-04T12:19:08.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14edae4640 1 -- 172.21.15.52:0/2869155806 learned_addr learned my addr 172.21.15.52:0/2869155806 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:08.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14ed2e3640 1 --2- 172.21.15.52:0/2869155806 >> v2:172.21.15.111:3300/0 conn(0x7f14e815bf50 0x7f14e815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14edae4640 1 -- 172.21.15.52:0/2869155806 >> v2:172.21.15.111:3300/0 conn(0x7f14e815bf50 msgr2=0x7f14e815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:08.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14edae4640 1 --2- 172.21.15.52:0/2869155806 >> v2:172.21.15.111:3300/0 conn(0x7f14e815bf50 0x7f14e815c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14edae4640 1 -- 172.21.15.52:0/2869155806 >> v1:172.21.15.111:6789/0 conn(0x7f14e80aa0c0 legacy=0x7f14e80ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:08.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14edae4640 1 -- 172.21.15.52:0/2869155806 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e806da50 con 0x7f14e80cd260 2024-05-04T12:19:08.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.653+0000 7f14ed2e3640 1 --2- 172.21.15.52:0/2869155806 >> v2:172.21.15.111:3300/0 conn(0x7f14e815bf50 0x7f14e815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:08.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14edae4640 1 --2- 172.21.15.52:0/2869155806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e80cd260 0x7f14e806e250 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f14e8069260 tx=0x7f14e001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3d2482f35e02ac server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:08.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14dffff640 1 -- 172.21.15.52:0/2869155806 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e00a5860 con 0x7f14e80cd260 2024-05-04T12:19:08.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14dffff640 1 -- 172.21.15.52:0/2869155806 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e00a5a00 con 0x7f14e80cd260 2024-05-04T12:19:08.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14dffff640 1 -- 172.21.15.52:0/2869155806 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e00a5dd0 con 0x7f14e80cd260 2024-05-04T12:19:08.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14ef547640 1 -- 172.21.15.52:0/2869155806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e80cd260 msgr2=0x7f14e806e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14ef547640 1 --2- 172.21.15.52:0/2869155806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e80cd260 0x7f14e806e250 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f14e8069260 tx=0x7f14e001c980 comp rx=0 tx=0).stop 2024-05-04T12:19:08.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14ef547640 1 -- 172.21.15.52:0/2869155806 shutdown_connections 2024-05-04T12:19:08.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14ef547640 1 --2- 172.21.15.52:0/2869155806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e80cd260 0x7f14e806e250 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14ef547640 1 --2- 172.21.15.52:0/2869155806 >> v2:172.21.15.111:3300/0 conn(0x7f14e815bf50 0x7f14e815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14ef547640 1 -- 172.21.15.52:0/2869155806 >> 172.21.15.52:0/2869155806 conn(0x7f14e80b14a0 msgr2=0x7f14e80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:08.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14ef547640 1 -- 172.21.15.52:0/2869155806 shutdown_connections 2024-05-04T12:19:08.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14ef547640 1 -- 172.21.15.52:0/2869155806 wait complete. 2024-05-04T12:19:08.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14ef547640 1 Processor -- start 2024-05-04T12:19:08.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14ef547640 1 -- start start 2024-05-04T12:19:08.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.657+0000 7f14ef547640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14e80aa0c0 0x7f14e8155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ef547640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e80cd260 0x7f14e80a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ef547640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e815bf50 0x7f14e8146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ef547640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81691b0 con 0x7f14e815bf50 2024-05-04T12:19:08.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ef547640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f14e8169030 con 0x7f14e80cd260 2024-05-04T12:19:08.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ef547640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f14e8169330 con 0x7f14e80aa0c0 2024-05-04T12:19:08.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14edae4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e815bf50 0x7f14e8146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14edae4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e815bf50 0x7f14e8146120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56542/0 (socket says 172.21.15.52:56542) 2024-05-04T12:19:08.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14edae4640 1 -- 172.21.15.52:0/4050799311 learned_addr learned my addr 172.21.15.52:0/4050799311 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:08.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ecae2640 1 --2- 172.21.15.52:0/4050799311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e80cd260 0x7f14e80a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ed2e3640 1 --2- 172.21.15.52:0/4050799311 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14e80aa0c0 0x7f14e8155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14edae4640 1 -- 172.21.15.52:0/4050799311 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14e80aa0c0 msgr2=0x7f14e8155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14edae4640 1 --2- 172.21.15.52:0/4050799311 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14e80aa0c0 0x7f14e8155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14edae4640 1 -- 172.21.15.52:0/4050799311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e80cd260 msgr2=0x7f14e80a96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14edae4640 1 --2- 172.21.15.52:0/4050799311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e80cd260 0x7f14e80a96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14edae4640 1 -- 172.21.15.52:0/4050799311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e81584b0 con 0x7f14e815bf50 2024-05-04T12:19:08.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14edae4640 1 --2- 172.21.15.52:0/4050799311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e815bf50 0x7f14e8146120 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f14e001b030 tx=0x7f14e0019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:08.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ddffb640 1 -- 172.21.15.52:0/4050799311 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e00b8880 con 0x7f14e815bf50 2024-05-04T12:19:08.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ddffb640 1 -- 172.21.15.52:0/4050799311 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e00be070 con 0x7f14e815bf50 2024-05-04T12:19:08.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ddffb640 1 -- 172.21.15.52:0/4050799311 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e00b8c60 con 0x7f14e815bf50 2024-05-04T12:19:08.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ef547640 1 -- 172.21.15.52:0/4050799311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e8158790 con 0x7f14e815bf50 2024-05-04T12:19:08.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ef547640 1 -- 172.21.15.52:0/4050799311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e8158d50 con 0x7f14e815bf50 2024-05-04T12:19:08.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ef547640 1 -- 172.21.15.52:0/4050799311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e8062e50 con 0x7f14e815bf50 2024-05-04T12:19:08.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ddffb640 1 -- 172.21.15.52:0/4050799311 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f14e00c3460 con 0x7f14e815bf50 2024-05-04T12:19:08.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ddffb640 1 --2- 172.21.15.52:0/4050799311 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f14cc0678c0 0x7f14cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:08.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ddffb640 1 -- 172.21.15.52:0/4050799311 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f14e0120d90 con 0x7f14e815bf50 2024-05-04T12:19:08.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ed2e3640 1 --2- 172.21.15.52:0/4050799311 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f14cc0678c0 0x7f14cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:08.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.661+0000 7f14ed2e3640 1 --2- 172.21.15.52:0/4050799311 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f14cc0678c0 0x7f14cc069d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f14e4006fd0 tx=0x7f14e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:08.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.665+0000 7f14ddffb640 1 -- 172.21.15.52:0/4050799311 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f14e00c8020 con 0x7f14e815bf50 2024-05-04T12:19:08.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.833+0000 7f14ef547640 1 -- 172.21.15.52:0/4050799311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f14e8075320 con 0x7f14e815bf50 2024-05-04T12:19:08.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.833+0000 7f14ddffb640 1 -- 172.21.15.52:0/4050799311 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f14e00165a0 con 0x7f14e815bf50 2024-05-04T12:19:08.837 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-04T12:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.837+0000 7f14ef547640 1 -- 172.21.15.52:0/4050799311 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f14cc0678c0 msgr2=0x7f14cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.837+0000 7f14ef547640 1 --2- 172.21.15.52:0/4050799311 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f14cc0678c0 0x7f14cc069d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f14e4006fd0 tx=0x7f14e4008040 comp rx=0 tx=0).stop 2024-05-04T12:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.837+0000 7f14ef547640 1 -- 172.21.15.52:0/4050799311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e815bf50 msgr2=0x7f14e8146120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.837+0000 7f14ef547640 1 --2- 172.21.15.52:0/4050799311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e815bf50 0x7f14e8146120 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f14e001b030 tx=0x7f14e0019720 comp rx=0 tx=0).stop 2024-05-04T12:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.837+0000 7f14ef547640 1 -- 172.21.15.52:0/4050799311 shutdown_connections 2024-05-04T12:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.837+0000 7f14ef547640 1 --2- 172.21.15.52:0/4050799311 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f14cc0678c0 0x7f14cc069d80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.837+0000 7f14ef547640 1 --2- 172.21.15.52:0/4050799311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14e815bf50 0x7f14e8146120 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.837+0000 7f14ef547640 1 --2- 172.21.15.52:0/4050799311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e80cd260 0x7f14e80a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.837+0000 7f14ef547640 1 --2- 172.21.15.52:0/4050799311 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14e80aa0c0 0x7f14e8155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.837+0000 7f14ef547640 1 -- 172.21.15.52:0/4050799311 >> 172.21.15.52:0/4050799311 conn(0x7f14e80b14a0 msgr2=0x7f14e8152260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.837+0000 7f14ef547640 1 -- 172.21.15.52:0/4050799311 shutdown_connections 2024-05-04T12:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:08.837+0000 7f14ef547640 1 -- 172.21.15.52:0/4050799311 wait complete. 2024-05-04T12:19:08.855 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2024-05-04T12:19:08.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:08.856 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-05-04T12:19:08.870 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T12:19:08.871 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T12:19:08.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:08.871 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-04T12:19:08.919 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-04T12:19:08.919 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T12:19:09.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.029+0000 7fb3e8cf3640 1 Processor -- start 2024-05-04T12:19:09.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.029+0000 7fb3e8cf3640 1 -- start start 2024-05-04T12:19:09.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e8cf3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3e415cf90 0x7fb3e40b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e8cf3640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3e4070de0 0x7fb3e406dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e8cf3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e415fe90 con 0x7fb3e415d370 2024-05-04T12:19:09.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e8cf3640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb3e4160170 con 0x7fb3e4070de0 2024-05-04T12:19:09.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e8cf3640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fb3e4160450 con 0x7fb3e415cf90 2024-05-04T12:19:09.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e2d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3e4070de0 0x7fb3e406dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e2d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3e4070de0 0x7fb3e406dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56554/0 (socket says 172.21.15.52:56554) 2024-05-04T12:19:09.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e2d9d640 1 -- 172.21.15.52:0/2780339514 learned_addr learned my addr 172.21.15.52:0/2780339514 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:09.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e2d9d640 1 -- 172.21.15.52:0/2780339514 >> v2:172.21.15.111:3300/0 conn(0x7fb3e415cf90 msgr2=0x7fb3e40b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:09.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e259c640 1 --2- 172.21.15.52:0/2780339514 >> v2:172.21.15.111:3300/0 conn(0x7fb3e415cf90 0x7fb3e40b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e2d9d640 1 --2- 172.21.15.52:0/2780339514 >> v2:172.21.15.111:3300/0 conn(0x7fb3e415cf90 0x7fb3e40b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e2d9d640 1 -- 172.21.15.52:0/2780339514 >> v1:172.21.15.111:6789/0 conn(0x7fb3e415d370 legacy=0x7fb3e40706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:09.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e2d9d640 1 -- 172.21.15.52:0/2780339514 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e4160730 con 0x7fb3e4070de0 2024-05-04T12:19:09.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e259c640 1 --2- 172.21.15.52:0/2780339514 >> v2:172.21.15.111:3300/0 conn(0x7fb3e415cf90 0x7fb3e40b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:09.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e2d9d640 1 --2- 172.21.15.52:0/2780339514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e4070de0 0x7fb3e406dea0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4071360 tx=0x7fb3dc01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c783e6302c1e9b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:09.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e159a640 1 -- 172.21.15.52:0/2780339514 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc0a5780 con 0x7fb3e4070de0 2024-05-04T12:19:09.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e159a640 1 -- 172.21.15.52:0/2780339514 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3dc0a5920 con 0x7fb3e4070de0 2024-05-04T12:19:09.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.033+0000 7fb3e159a640 1 -- 172.21.15.52:0/2780339514 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc0a5cf0 con 0x7fb3e4070de0 2024-05-04T12:19:09.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/2780339514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e4070de0 msgr2=0x7fb3e406dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 --2- 172.21.15.52:0/2780339514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e4070de0 0x7fb3e406dea0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4071360 tx=0x7fb3dc01c9b0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/2780339514 shutdown_connections 2024-05-04T12:19:09.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 --2- 172.21.15.52:0/2780339514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e4070de0 0x7fb3e406dea0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 --2- 172.21.15.52:0/2780339514 >> v2:172.21.15.111:3300/0 conn(0x7fb3e415cf90 0x7fb3e40b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/2780339514 >> 172.21.15.52:0/2780339514 conn(0x7fb3e4067070 msgr2=0x7fb3e4067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:09.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/2780339514 shutdown_connections 2024-05-04T12:19:09.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/2780339514 wait complete. 2024-05-04T12:19:09.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 Processor -- start 2024-05-04T12:19:09.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 -- start start 2024-05-04T12:19:09.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3e4070de0 0x7fb3e4154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e415cf90 0x7fb3e4140c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3e415d370 0x7fb3e415b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e41693b0 con 0x7fb3e415cf90 2024-05-04T12:19:09.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e4169230 con 0x7fb3e415d370 2024-05-04T12:19:09.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e4169530 con 0x7fb3e4070de0 2024-05-04T12:19:09.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e1d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e415cf90 0x7fb3e4140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e1d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e415cf90 0x7fb3e4140c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56562/0 (socket says 172.21.15.52:56562) 2024-05-04T12:19:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e1d9b640 1 -- 172.21.15.52:0/4195049743 learned_addr learned my addr 172.21.15.52:0/4195049743 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e259c640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3e4070de0 0x7fb3e4154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e2d9d640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3e415d370 0x7fb3e415b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e1d9b640 1 -- 172.21.15.52:0/4195049743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3e4070de0 msgr2=0x7fb3e4154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e1d9b640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3e4070de0 0x7fb3e4154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e1d9b640 1 -- 172.21.15.52:0/4195049743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3e415d370 msgr2=0x7fb3e415b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e1d9b640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3e415d370 0x7fb3e415b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e1d9b640 1 -- 172.21.15.52:0/4195049743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e4072850 con 0x7fb3e415cf90 2024-05-04T12:19:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e259c640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3e4070de0 0x7fb3e4154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e2d9d640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3e415d370 0x7fb3e415b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e1d9b640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e415cf90 0x7fb3e4140c50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800b410 tx=0x7fb3d800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3d2ffd640 1 -- 172.21.15.52:0/4195049743 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8015030 con 0x7fb3e415cf90 2024-05-04T12:19:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3d2ffd640 1 -- 172.21.15.52:0/4195049743 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d8004020 con 0x7fb3e415cf90 2024-05-04T12:19:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/4195049743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e413e250 con 0x7fb3e415cf90 2024-05-04T12:19:09.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3d2ffd640 1 -- 172.21.15.52:0/4195049743 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8004410 con 0x7fb3e415cf90 2024-05-04T12:19:09.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/4195049743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e4072290 con 0x7fb3e415cf90 2024-05-04T12:19:09.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.037+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/4195049743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e4062e50 con 0x7fb3e415cf90 2024-05-04T12:19:09.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.041+0000 7fb3d2ffd640 1 -- 172.21.15.52:0/4195049743 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb3d8018070 con 0x7fb3e415cf90 2024-05-04T12:19:09.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.041+0000 7fb3d2ffd640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3c40678f0 0x7fb3c4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.041+0000 7fb3d2ffd640 1 -- 172.21.15.52:0/4195049743 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7fb3d80857f0 con 0x7fb3e415cf90 2024-05-04T12:19:09.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.041+0000 7fb3e259c640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3c40678f0 0x7fb3c4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.041+0000 7fb3e259c640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3c40678f0 0x7fb3c4069db0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4008930 tx=0x7fb3d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:09.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.045+0000 7fb3d2ffd640 1 -- 172.21.15.52:0/4195049743 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb3d80562a0 con 0x7fb3e415cf90 2024-05-04T12:19:09.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.237+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/4195049743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3e4075320 con 0x7fb3e415cf90 2024-05-04T12:19:09.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.237+0000 7fb3d2ffd640 1 -- 172.21.15.52:0/4195049743 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7fb3d8055ba0 con 0x7fb3e415cf90 2024-05-04T12:19:09.242 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:09.242 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6836/482177111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":482177111},{"type":"v1","addr":"172.21.15.52:6836","nonce":482177111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/163453709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6835","nonce":163453709},{"type":"v1","addr":"172.21.15.52:6837","nonce":163453709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3596668218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3596668218},{"type":"v1","addr":"172.21.15.111:6833","nonce":3596668218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-04T12:18:22.189525+0000","modified":"2024-05-04T12:18:25.240402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/1656435333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1656435333},{"type":"v1","addr":"172.21.15.111:6835","nonce":1656435333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-05-04T12:19:09.243 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 7 2024-05-04T12:19:09.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.241+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/4195049743 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3c40678f0 msgr2=0x7fb3c4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.241+0000 7fb3e8cf3640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3c40678f0 0x7fb3c4069db0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4008930 tx=0x7fb3d4008040 comp rx=0 tx=0).stop 2024-05-04T12:19:09.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.241+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/4195049743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e415cf90 msgr2=0x7fb3e4140c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.241+0000 7fb3e8cf3640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e415cf90 0x7fb3e4140c50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800b410 tx=0x7fb3d800b8d0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.241+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/4195049743 shutdown_connections 2024-05-04T12:19:09.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.241+0000 7fb3e8cf3640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3c40678f0 0x7fb3c4069db0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.241+0000 7fb3e8cf3640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3e415d370 0x7fb3e415b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.241+0000 7fb3e8cf3640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e415cf90 0x7fb3e4140c50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.241+0000 7fb3e8cf3640 1 --2- 172.21.15.52:0/4195049743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3e4070de0 0x7fb3e4154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.241+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/4195049743 >> 172.21.15.52:0/4195049743 conn(0x7fb3e4067070 msgr2=0x7fb3e41561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:09.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.241+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/4195049743 shutdown_connections 2024-05-04T12:19:09.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.241+0000 7fb3e8cf3640 1 -- 172.21.15.52:0/4195049743 wait complete. 2024-05-04T12:19:09.260 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:19:09.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.369+0000 7f79053d7640 1 Processor -- start 2024-05-04T12:19:09.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.369+0000 7f79053d7640 1 -- start start 2024-05-04T12:19:09.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.369+0000 7f79053d7640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7900144eb0 0x7f79000d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.369+0000 7f79053d7640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f790006fc70 con 0x7f79001483f0 2024-05-04T12:19:09.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.369+0000 7f79053d7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7900162a60 con 0x7f79000c9370 2024-05-04T12:19:09.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.369+0000 7f79053d7640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f7900162d40 con 0x7f7900144eb0 2024-05-04T12:19:09.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f78feffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7900144eb0 0x7f79000d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f78fe7fc640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f79001483f0 0x7f790006ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46856/0 (socket says 172.21.15.52:46856) 2024-05-04T12:19:09.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f78feffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7900144eb0 0x7f79000d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56570/0 (socket says 172.21.15.52:56570) 2024-05-04T12:19:09.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f78fe7fc640 1 -- 172.21.15.52:0/2441742726 learned_addr learned my addr 172.21.15.52:0/2441742726 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:09.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f78feffd640 1 -- 172.21.15.52:0/2441742726 >> v1:172.21.15.111:6789/0 conn(0x7f79000c9370 legacy=0x7f790006f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:09.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f78feffd640 1 -- 172.21.15.52:0/2441742726 >> v1:172.21.15.52:6789/0 conn(0x7f79001483f0 legacy=0x7f790006ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f78fdffb640 1 -- 172.21.15.52:0/2441742726 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4110262090 0 0) 0x7f790006fc70 con 0x7f79001483f0 2024-05-04T12:19:09.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f78feffd640 1 -- 172.21.15.52:0/2441742726 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7900163020 con 0x7f7900144eb0 2024-05-04T12:19:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f78feffd640 1 --2- 172.21.15.52:0/2441742726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7900144eb0 0x7f79000d2380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f78f00097c0 tx=0x7f78f001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4618fb930bf63ffe server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f78fdffb640 1 -- 172.21.15.52:0/2441742726 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f00b0070 con 0x7f7900144eb0 2024-05-04T12:19:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f78fdffb640 1 -- 172.21.15.52:0/2441742726 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f00ab600 con 0x7f7900144eb0 2024-05-04T12:19:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f78fdffb640 1 -- 172.21.15.52:0/2441742726 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f00ab9d0 con 0x7f7900144eb0 2024-05-04T12:19:09.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f79053d7640 1 -- 172.21.15.52:0/2441742726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7900144eb0 msgr2=0x7f79000d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f79053d7640 1 --2- 172.21.15.52:0/2441742726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7900144eb0 0x7f79000d2380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f78f00097c0 tx=0x7f78f001c460 comp rx=0 tx=0).stop 2024-05-04T12:19:09.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f79053d7640 1 -- 172.21.15.52:0/2441742726 shutdown_connections 2024-05-04T12:19:09.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f79053d7640 1 --2- 172.21.15.52:0/2441742726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7900144eb0 0x7f79000d2380 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f79053d7640 1 -- 172.21.15.52:0/2441742726 >> 172.21.15.52:0/2441742726 conn(0x7f790006b210 msgr2=0x7f790006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:09.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f79053d7640 1 -- 172.21.15.52:0/2441742726 shutdown_connections 2024-05-04T12:19:09.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.373+0000 7f79053d7640 1 -- 172.21.15.52:0/2441742726 wait complete. 2024-05-04T12:19:09.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f79053d7640 1 Processor -- start 2024-05-04T12:19:09.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f79053d7640 1 -- start start 2024-05-04T12:19:09.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f79053d7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79000c9370 0x7f790013f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f79053d7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7900144eb0 0x7f79001611c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f79053d7640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79001483f0 0x7f790015e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f79053d7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79000d2cd0 con 0x7f7900144eb0 2024-05-04T12:19:09.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f79053d7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f79000d2b50 con 0x7f79000c9370 2024-05-04T12:19:09.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f79053d7640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f790016d150 con 0x7f79001483f0 2024-05-04T12:19:09.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78fe7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7900144eb0 0x7f79001611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78fe7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7900144eb0 0x7f79001611c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56586/0 (socket says 172.21.15.52:56586) 2024-05-04T12:19:09.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78fe7fc640 1 -- 172.21.15.52:0/1948565419 learned_addr learned my addr 172.21.15.52:0/1948565419 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:09.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78feffd640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79000c9370 0x7f790013f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78ff7fe640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79001483f0 0x7f790015e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78fe7fc640 1 -- 172.21.15.52:0/1948565419 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79001483f0 msgr2=0x7f790015e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78fe7fc640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79001483f0 0x7f790015e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78fe7fc640 1 -- 172.21.15.52:0/1948565419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79000c9370 msgr2=0x7f790013f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78fe7fc640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79000c9370 0x7f790013f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78fe7fc640 1 -- 172.21.15.52:0/1948565419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790013fb90 con 0x7f7900144eb0 2024-05-04T12:19:09.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78feffd640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79000c9370 0x7f790013f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:09.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78fe7fc640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7900144eb0 0x7f79001611c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f78f400c480 tx=0x7f78f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:09.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78e77fe640 1 -- 172.21.15.52:0/1948565419 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f4018070 con 0x7f7900144eb0 2024-05-04T12:19:09.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78e77fe640 1 -- 172.21.15.52:0/1948565419 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f4007500 con 0x7f7900144eb0 2024-05-04T12:19:09.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78e77fe640 1 -- 172.21.15.52:0/1948565419 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f4004a30 con 0x7f7900144eb0 2024-05-04T12:19:09.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f79053d7640 1 -- 172.21.15.52:0/1948565419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7900160400 con 0x7f7900144eb0 2024-05-04T12:19:09.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f79053d7640 1 -- 172.21.15.52:0/1948565419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7900160ca0 con 0x7f7900144eb0 2024-05-04T12:19:09.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f79053d7640 1 -- 172.21.15.52:0/1948565419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79001605d0 con 0x7f7900144eb0 2024-05-04T12:19:09.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78e77fe640 1 -- 172.21.15.52:0/1948565419 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f78f4004bd0 con 0x7f7900144eb0 2024-05-04T12:19:09.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78e77fe640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f78ec0678c0 0x7f78ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78e77fe640 1 -- 172.21.15.52:0/1948565419 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f78f4084ef0 con 0x7f7900144eb0 2024-05-04T12:19:09.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.377+0000 7f78feffd640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f78ec0678c0 0x7f78ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.381+0000 7f78feffd640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f78ec0678c0 0x7f78ec069d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f78f00ad000 tx=0x7f78f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:09.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.385+0000 7f78e77fe640 1 -- 172.21.15.52:0/1948565419 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f78f4055a30 con 0x7f7900144eb0 2024-05-04T12:19:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.541+0000 7f79053d7640 1 -- 172.21.15.52:0/1948565419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7900062e50 con 0x7f7900144eb0 2024-05-04T12:19:09.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.541+0000 7f78e77fe640 1 -- 172.21.15.52:0/1948565419 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16913 (secure 0 0 0) 0x7f78f4055330 con 0x7f7900144eb0 2024-05-04T12:19:09.546 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:09.546 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:19:04.046290+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:19:09.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.545+0000 7f79053d7640 1 -- 172.21.15.52:0/1948565419 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f78ec0678c0 msgr2=0x7f78ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.545+0000 7f79053d7640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f78ec0678c0 0x7f78ec069d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f78f00ad000 tx=0x7f78f0008040 comp rx=0 tx=0).stop 2024-05-04T12:19:09.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.545+0000 7f79053d7640 1 -- 172.21.15.52:0/1948565419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7900144eb0 msgr2=0x7f79001611c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.545+0000 7f79053d7640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7900144eb0 0x7f79001611c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f78f400c480 tx=0x7f78f400c940 comp rx=0 tx=0).stop 2024-05-04T12:19:09.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.545+0000 7f79053d7640 1 -- 172.21.15.52:0/1948565419 shutdown_connections 2024-05-04T12:19:09.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.545+0000 7f79053d7640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f78ec0678c0 0x7f78ec069d80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.545+0000 7f79053d7640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79001483f0 0x7f790015e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.545+0000 7f79053d7640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7900144eb0 0x7f79001611c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.545+0000 7f79053d7640 1 --2- 172.21.15.52:0/1948565419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79000c9370 0x7f790013f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.545+0000 7f79053d7640 1 -- 172.21.15.52:0/1948565419 >> 172.21.15.52:0/1948565419 conn(0x7f790006b210 msgr2=0x7f79000ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:09.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.545+0000 7f79053d7640 1 -- 172.21.15.52:0/1948565419 shutdown_connections 2024-05-04T12:19:09.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.545+0000 7f79053d7640 1 -- 172.21.15.52:0/1948565419 wait complete. 2024-05-04T12:19:09.563 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:18:20.636173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:18:21.180614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:19:09.564 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:19:09.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2fde1640 1 Processor -- start 2024-05-04T12:19:09.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2fde1640 1 -- start start 2024-05-04T12:19:09.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2fde1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8f28147280 0x7f8f2806d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2fde1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8f2806cdd0 0x7f8f280d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2fde1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f28072b90 con 0x7f8f2815ea70 2024-05-04T12:19:09.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2fde1640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f8f28147660 con 0x7f8f28147280 2024-05-04T12:19:09.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2fde1640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f8f280725d0 con 0x7f8f2806cdd0 2024-05-04T12:19:09.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2db7d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8f28147280 0x7f8f2806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2d37c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f8f2815ea70 0x7f8f2806c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46864/0 (socket says 172.21.15.52:46864) 2024-05-04T12:19:09.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2d37c640 1 -- 172.21.15.52:0/2116501156 learned_addr learned my addr 172.21.15.52:0/2116501156 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:09.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2e37e640 1 --2- 172.21.15.52:0/2116501156 >> v2:172.21.15.111:3300/0 conn(0x7f8f2806cdd0 0x7f8f280d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2cb7b640 1 -- 172.21.15.52:0/2116501156 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2152574611 0 0) 0x7f8f18001120 con 0x7f8f2815ea70 2024-05-04T12:19:09.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2cb7b640 1 -- 172.21.15.52:0/2116501156 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f14003530 con 0x7f8f2815ea70 2024-05-04T12:19:09.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2db7d640 1 -- 172.21.15.52:0/2116501156 >> v2:172.21.15.111:3300/0 conn(0x7f8f2806cdd0 msgr2=0x7f8f280d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:09.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2db7d640 1 --2- 172.21.15.52:0/2116501156 >> v2:172.21.15.111:3300/0 conn(0x7f8f2806cdd0 0x7f8f280d0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2db7d640 1 -- 172.21.15.52:0/2116501156 >> v1:172.21.15.52:6789/0 conn(0x7f8f2815ea70 legacy=0x7f8f2806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2cb7b640 1 -- 172.21.15.52:0/2116501156 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1818297147 0 0) 0x7f8f14003530 con 0x7f8f2815ea70 2024-05-04T12:19:09.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2db7d640 1 -- 172.21.15.52:0/2116501156 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f280728b0 con 0x7f8f28147280 2024-05-04T12:19:09.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.677+0000 7f8f2e37e640 1 --2- 172.21.15.52:0/2116501156 >> v2:172.21.15.111:3300/0 conn(0x7f8f2806cdd0 0x7f8f280d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:09.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.681+0000 7f8f2db7d640 1 --2- 172.21.15.52:0/2116501156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f28147280 0x7f8f2806d400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8f2806ee10 tx=0x7f8f2401c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272805f9cca11215 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:09.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.681+0000 7f8f2cb7b640 1 -- 172.21.15.52:0/2116501156 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f240b0070 con 0x7f8f28147280 2024-05-04T12:19:09.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.681+0000 7f8f2cb7b640 1 -- 172.21.15.52:0/2116501156 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f240ab400 con 0x7f8f28147280 2024-05-04T12:19:09.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.681+0000 7f8f2cb7b640 1 -- 172.21.15.52:0/2116501156 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18001850 con 0x7f8f28147280 2024-05-04T12:19:09.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.681+0000 7f8f2fde1640 1 -- 172.21.15.52:0/2116501156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f28147280 msgr2=0x7f8f2806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.681+0000 7f8f2fde1640 1 --2- 172.21.15.52:0/2116501156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f28147280 0x7f8f2806d400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8f2806ee10 tx=0x7f8f2401c370 comp rx=0 tx=0).stop 2024-05-04T12:19:09.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.681+0000 7f8f2fde1640 1 -- 172.21.15.52:0/2116501156 shutdown_connections 2024-05-04T12:19:09.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.681+0000 7f8f2fde1640 1 --2- 172.21.15.52:0/2116501156 >> v2:172.21.15.111:3300/0 conn(0x7f8f2806cdd0 0x7f8f280d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.681+0000 7f8f2fde1640 1 --2- 172.21.15.52:0/2116501156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f28147280 0x7f8f2806d400 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.681+0000 7f8f2fde1640 1 -- 172.21.15.52:0/2116501156 >> 172.21.15.52:0/2116501156 conn(0x7f8f2806b210 msgr2=0x7f8f2806b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:09.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.681+0000 7f8f2fde1640 1 -- 172.21.15.52:0/2116501156 shutdown_connections 2024-05-04T12:19:09.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.681+0000 7f8f2fde1640 1 -- 172.21.15.52:0/2116501156 wait complete. 2024-05-04T12:19:09.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2fde1640 1 Processor -- start 2024-05-04T12:19:09.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2fde1640 1 -- start start 2024-05-04T12:19:09.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2fde1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f2806cdd0 0x7f8f2813de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2fde1640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f28147280 0x7f8f28145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2fde1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f2815ea70 0x7f8f2815a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2fde1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281480a0 con 0x7f8f2815ea70 2024-05-04T12:19:09.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2fde1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28147f20 con 0x7f8f2806cdd0 2024-05-04T12:19:09.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2fde1640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f8f280721c0 con 0x7f8f28147280 2024-05-04T12:19:09.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2e37e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f2815ea70 0x7f8f2815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2e37e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f2815ea70 0x7f8f2815a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56594/0 (socket says 172.21.15.52:56594) 2024-05-04T12:19:09.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2e37e640 1 -- 172.21.15.52:0/865524567 learned_addr learned my addr 172.21.15.52:0/865524567 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:09.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2db7d640 1 --2- 172.21.15.52:0/865524567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f2806cdd0 0x7f8f2813de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2d37c640 1 --2- 172.21.15.52:0/865524567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f28147280 0x7f8f28145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2e37e640 1 -- 172.21.15.52:0/865524567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f28147280 msgr2=0x7f8f28145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2e37e640 1 --2- 172.21.15.52:0/865524567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f28147280 0x7f8f28145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2e37e640 1 -- 172.21.15.52:0/865524567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f2806cdd0 msgr2=0x7f8f2813de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2e37e640 1 --2- 172.21.15.52:0/865524567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f2806cdd0 0x7f8f2813de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2e37e640 1 -- 172.21.15.52:0/865524567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2815aea0 con 0x7f8f2815ea70 2024-05-04T12:19:09.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2e37e640 1 --2- 172.21.15.52:0/865524567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f2815ea70 0x7f8f2815a030 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8f20007ef0 tx=0x7f8f2000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:09.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f1e7fc640 1 -- 172.21.15.52:0/865524567 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2001a070 con 0x7f8f2815ea70 2024-05-04T12:19:09.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f1e7fc640 1 -- 172.21.15.52:0/865524567 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f20003ba0 con 0x7f8f2815ea70 2024-05-04T12:19:09.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f1e7fc640 1 -- 172.21.15.52:0/865524567 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20004030 con 0x7f8f2815ea70 2024-05-04T12:19:09.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2fde1640 1 -- 172.21.15.52:0/865524567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2815b180 con 0x7f8f2815ea70 2024-05-04T12:19:09.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2fde1640 1 -- 172.21.15.52:0/865524567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2815b740 con 0x7f8f2815ea70 2024-05-04T12:19:09.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f2fde1640 1 -- 172.21.15.52:0/865524567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2815b940 con 0x7f8f2815ea70 2024-05-04T12:19:09.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.685+0000 7f8f1e7fc640 1 -- 172.21.15.52:0/865524567 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8f2001d070 con 0x7f8f2815ea70 2024-05-04T12:19:09.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.689+0000 7f8f1e7fc640 1 --2- 172.21.15.52:0/865524567 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8f0c0678c0 0x7f8f0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:09.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.689+0000 7f8f1e7fc640 1 -- 172.21.15.52:0/865524567 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f8f20085f40 con 0x7f8f2815ea70 2024-05-04T12:19:09.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.689+0000 7f8f2db7d640 1 --2- 172.21.15.52:0/865524567 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8f0c0678c0 0x7f8f0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:09.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.689+0000 7f8f2db7d640 1 --2- 172.21.15.52:0/865524567 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8f0c0678c0 0x7f8f0c069d80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8f240ad000 tx=0x7f8f240a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:09.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.693+0000 7f8f1e7fc640 1 -- 172.21.15.52:0/865524567 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8f2815b940 con 0x7f8f2815ea70 2024-05-04T12:19:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.853+0000 7f8f2fde1640 1 -- 172.21.15.52:0/865524567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8f28062e50 con 0x7f8f2815ea70 2024-05-04T12:19:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.857+0000 7f8f1e7fc640 1 -- 172.21.15.52:0/865524567 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f8f20010070 con 0x7f8f2815ea70 2024-05-04T12:19:09.860 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:19:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.857+0000 7f8f2fde1640 1 -- 172.21.15.52:0/865524567 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8f0c0678c0 msgr2=0x7f8f0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.857+0000 7f8f2fde1640 1 --2- 172.21.15.52:0/865524567 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8f0c0678c0 0x7f8f0c069d80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8f240ad000 tx=0x7f8f240a6000 comp rx=0 tx=0).stop 2024-05-04T12:19:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.857+0000 7f8f2fde1640 1 -- 172.21.15.52:0/865524567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f2815ea70 msgr2=0x7f8f2815a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.857+0000 7f8f2fde1640 1 --2- 172.21.15.52:0/865524567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f2815ea70 0x7f8f2815a030 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8f20007ef0 tx=0x7f8f2000d4a0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.857+0000 7f8f2fde1640 1 -- 172.21.15.52:0/865524567 shutdown_connections 2024-05-04T12:19:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.857+0000 7f8f2fde1640 1 --2- 172.21.15.52:0/865524567 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f8f0c0678c0 0x7f8f0c069d80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.857+0000 7f8f2fde1640 1 --2- 172.21.15.52:0/865524567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f2815ea70 0x7f8f2815a030 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.857+0000 7f8f2fde1640 1 --2- 172.21.15.52:0/865524567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8f28147280 0x7f8f28145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.857+0000 7f8f2fde1640 1 --2- 172.21.15.52:0/865524567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f2806cdd0 0x7f8f2813de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.857+0000 7f8f2fde1640 1 -- 172.21.15.52:0/865524567 >> 172.21.15.52:0/865524567 conn(0x7f8f2806b210 msgr2=0x7f8f281612a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.857+0000 7f8f2fde1640 1 -- 172.21.15.52:0/865524567 shutdown_connections 2024-05-04T12:19:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:09.857+0000 7f8f2fde1640 1 -- 172.21.15.52:0/865524567 wait complete. 2024-05-04T12:19:09.879 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T12:19:10.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f322683e640 1 Processor -- start 2024-05-04T12:19:10.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f322683e640 1 -- start start 2024-05-04T12:19:10.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f322683e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3220145960 0x7f3220160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f322683e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f322015c590 0x7f32200cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f322683e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f322006dbf0 con 0x7f32200cccd0 2024-05-04T12:19:10.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f322683e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f32200cd140 con 0x7f3220145960 2024-05-04T12:19:10.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f322683e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f3220160ad0 con 0x7f322015c590 2024-05-04T12:19:10.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f321f7fe640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f322015c590 0x7f32200cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f321ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3220145960 0x7f3220160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f321f7fe640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f322015c590 0x7f32200cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.52:55288/0 (socket says 172.21.15.52:55288) 2024-05-04T12:19:10.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f321f7fe640 1 -- 172.21.15.52:0/217453043 learned_addr learned my addr 172.21.15.52:0/217453043 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:10.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f321f7fe640 1 -- 172.21.15.52:0/217453043 >> v2:172.21.15.111:3300/0 conn(0x7f3220145960 msgr2=0x7f3220160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:10.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f321f7fe640 1 --2- 172.21.15.52:0/217453043 >> v2:172.21.15.111:3300/0 conn(0x7f3220145960 0x7f3220160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f321f7fe640 1 -- 172.21.15.52:0/217453043 >> v1:172.21.15.111:6789/0 conn(0x7f32200cccd0 legacy=0x7f32200cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.005+0000 7f321f7fe640 1 -- 172.21.15.52:0/217453043 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322006f090 con 0x7f322015c590 2024-05-04T12:19:10.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f321f7fe640 1 --2- 172.21.15.52:0/217453043 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f322015c590 0x7f32200cc790 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3208009e50 tx=0x7f320801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f17d031eca6beb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:10.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f321effd640 1 -- 172.21.15.52:0/217453043 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080a5750 con 0x7f322015c590 2024-05-04T12:19:10.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f321effd640 1 -- 172.21.15.52:0/217453043 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32080a58f0 con 0x7f322015c590 2024-05-04T12:19:10.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f321effd640 1 -- 172.21.15.52:0/217453043 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080a5cc0 con 0x7f322015c590 2024-05-04T12:19:10.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 -- 172.21.15.52:0/217453043 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f322015c590 msgr2=0x7f32200cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 --2- 172.21.15.52:0/217453043 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f322015c590 0x7f32200cc790 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3208009e50 tx=0x7f320801c970 comp rx=0 tx=0).stop 2024-05-04T12:19:10.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 -- 172.21.15.52:0/217453043 shutdown_connections 2024-05-04T12:19:10.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 --2- 172.21.15.52:0/217453043 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f322015c590 0x7f32200cc790 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 --2- 172.21.15.52:0/217453043 >> v2:172.21.15.111:3300/0 conn(0x7f3220145960 0x7f3220160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 -- 172.21.15.52:0/217453043 >> 172.21.15.52:0/217453043 conn(0x7f3220067730 msgr2=0x7f32200b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:10.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 -- 172.21.15.52:0/217453043 shutdown_connections 2024-05-04T12:19:10.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 -- 172.21.15.52:0/217453043 wait complete. 2024-05-04T12:19:10.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 Processor -- start 2024-05-04T12:19:10.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 -- start start 2024-05-04T12:19:10.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32200cccd0 0x7f32201449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3220145960 0x7f32201f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f322015c590 0x7f322013f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f32200cf890 con 0x7f3220145960 2024-05-04T12:19:10.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f322006d880 con 0x7f32200cccd0 2024-05-04T12:19:10.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f322683e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f322006cc40 con 0x7f322015c590 2024-05-04T12:19:10.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f321f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3220145960 0x7f32201f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.009+0000 7f321f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3220145960 0x7f32201f5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56598/0 (socket says 172.21.15.52:56598) 2024-05-04T12:19:10.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321f7fe640 1 -- 172.21.15.52:0/3855669939 learned_addr learned my addr 172.21.15.52:0/3855669939 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:10.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f3224ddb640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f322015c590 0x7f322013f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321ffff640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32200cccd0 0x7f32201449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321f7fe640 1 -- 172.21.15.52:0/3855669939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f322015c590 msgr2=0x7f322013f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321f7fe640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f322015c590 0x7f322013f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321f7fe640 1 -- 172.21.15.52:0/3855669939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32200cccd0 msgr2=0x7f32201449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321f7fe640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32200cccd0 0x7f32201449c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321f7fe640 1 -- 172.21.15.52:0/3855669939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32201f85b0 con 0x7f3220145960 2024-05-04T12:19:10.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f3224ddb640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f322015c590 0x7f322013f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:10.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321f7fe640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3220145960 0x7f32201f5a30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f32080af040 tx=0x7f3208019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:10.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321cff9640 1 -- 172.21.15.52:0/3855669939 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080bc070 con 0x7f3220145960 2024-05-04T12:19:10.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321cff9640 1 -- 172.21.15.52:0/3855669939 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32080b8660 con 0x7f3220145960 2024-05-04T12:19:10.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321cff9640 1 -- 172.21.15.52:0/3855669939 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080b8a50 con 0x7f3220145960 2024-05-04T12:19:10.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f322683e640 1 -- 172.21.15.52:0/3855669939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32201f8890 con 0x7f3220145960 2024-05-04T12:19:10.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f322683e640 1 -- 172.21.15.52:0/3855669939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32201f8e50 con 0x7f3220145960 2024-05-04T12:19:10.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f322683e640 1 -- 172.21.15.52:0/3855669939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3220062e50 con 0x7f3220145960 2024-05-04T12:19:10.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321cff9640 1 -- 172.21.15.52:0/3855669939 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f32080b8bf0 con 0x7f3220145960 2024-05-04T12:19:10.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321cff9640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f32040678c0 0x7f3204069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321cff9640 1 -- 172.21.15.52:0/3855669939 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f3208120980 con 0x7f3220145960 2024-05-04T12:19:10.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321ffff640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f32040678c0 0x7f3204069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.013+0000 7f321ffff640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f32040678c0 0x7f3204069d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3218009af0 tx=0x7f3218008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:10.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.017+0000 7f321cff9640 1 -- 172.21.15.52:0/3855669939 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3208016620 con 0x7f3220145960 2024-05-04T12:19:10.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.181+0000 7f322683e640 1 -- 172.21.15.52:0/3855669939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3220075320 con 0x7f3220145960 2024-05-04T12:19:10.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.181+0000 7f321cff9640 1 -- 172.21.15.52:0/3855669939 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f32080f1550 con 0x7f3220145960 2024-05-04T12:19:10.184 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T12:19:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.181+0000 7f322683e640 1 -- 172.21.15.52:0/3855669939 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f32040678c0 msgr2=0x7f3204069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.181+0000 7f322683e640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f32040678c0 0x7f3204069d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3218009af0 tx=0x7f3218008040 comp rx=0 tx=0).stop 2024-05-04T12:19:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.181+0000 7f322683e640 1 -- 172.21.15.52:0/3855669939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3220145960 msgr2=0x7f32201f5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.181+0000 7f322683e640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3220145960 0x7f32201f5a30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f32080af040 tx=0x7f3208019710 comp rx=0 tx=0).stop 2024-05-04T12:19:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.181+0000 7f322683e640 1 -- 172.21.15.52:0/3855669939 shutdown_connections 2024-05-04T12:19:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.181+0000 7f322683e640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f32040678c0 0x7f3204069d80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.181+0000 7f322683e640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f322015c590 0x7f322013f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.181+0000 7f322683e640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3220145960 0x7f32201f5a30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.181+0000 7f322683e640 1 --2- 172.21.15.52:0/3855669939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32200cccd0 0x7f32201449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.181+0000 7f322683e640 1 -- 172.21.15.52:0/3855669939 >> 172.21.15.52:0/3855669939 conn(0x7f3220067730 msgr2=0x7f322013e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.181+0000 7f322683e640 1 -- 172.21.15.52:0/3855669939 shutdown_connections 2024-05-04T12:19:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.181+0000 7f322683e640 1 -- 172.21.15.52:0/3855669939 wait complete. 2024-05-04T12:19:10.203 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T12:19:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4bb64ac640 1 Processor -- start 2024-05-04T12:19:10.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4bb64ac640 1 -- start start 2024-05-04T12:19:10.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4bb64ac640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4bb0145960 0x7f4bb0160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4bb64ac640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4bb00ccea0 0x7f4bb006e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4bb64ac640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb006dbf0 con 0x7f4bb015c590 2024-05-04T12:19:10.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4bb64ac640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f4bb0160ad0 con 0x7f4bb0145960 2024-05-04T12:19:10.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4bb64ac640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb006cfe0 con 0x7f4bb00ccea0 2024-05-04T12:19:10.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4baffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4bb0145960 0x7f4bb0160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4bb4a49640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4bb00ccea0 0x7f4bb006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4baffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4bb0145960 0x7f4bb0160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:34588/0 (socket says 172.21.15.52:34588) 2024-05-04T12:19:10.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4baffff640 1 -- 172.21.15.52:0/525271237 learned_addr learned my addr 172.21.15.52:0/525271237 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:10.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4baf7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4bb015c590 0x7f4bb00cc790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.52:34812/0 (socket says 172.21.15.52:34812) 2024-05-04T12:19:10.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4baffff640 1 -- 172.21.15.52:0/525271237 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4bb00ccea0 msgr2=0x7f4bb006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:10.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4baffff640 1 --2- 172.21.15.52:0/525271237 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4bb00ccea0 0x7f4bb006e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4baffff640 1 -- 172.21.15.52:0/525271237 >> v1:172.21.15.111:6789/0 conn(0x7f4bb015c590 legacy=0x7f4bb00cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.329+0000 7f4baffff640 1 -- 172.21.15.52:0/525271237 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb006d2c0 con 0x7f4bb0145960 2024-05-04T12:19:10.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.333+0000 7f4baffff640 1 --2- 172.21.15.52:0/525271237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb0145960 0x7f4bb0160590 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009870 tx=0x7f4ba401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=847d643f768dcc4d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:10.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.333+0000 7f4baeffd640 1 -- 172.21.15.52:0/525271237 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba40aa600 con 0x7f4bb0145960 2024-05-04T12:19:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.333+0000 7f4baeffd640 1 -- 172.21.15.52:0/525271237 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba40af070 con 0x7f4bb0145960 2024-05-04T12:19:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.333+0000 7f4baeffd640 1 -- 172.21.15.52:0/525271237 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba40aa9c0 con 0x7f4bb0145960 2024-05-04T12:19:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.333+0000 7f4bb64ac640 1 -- 172.21.15.52:0/525271237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb0145960 msgr2=0x7f4bb0160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.333+0000 7f4bb64ac640 1 --2- 172.21.15.52:0/525271237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb0145960 0x7f4bb0160590 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009870 tx=0x7f4ba401c4a0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.333+0000 7f4bb64ac640 1 -- 172.21.15.52:0/525271237 shutdown_connections 2024-05-04T12:19:10.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.333+0000 7f4bb64ac640 1 --2- 172.21.15.52:0/525271237 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4bb00ccea0 0x7f4bb006e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.333+0000 7f4bb64ac640 1 --2- 172.21.15.52:0/525271237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb0145960 0x7f4bb0160590 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.333+0000 7f4bb64ac640 1 -- 172.21.15.52:0/525271237 >> 172.21.15.52:0/525271237 conn(0x7f4bb0067730 msgr2=0x7f4bb00b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:10.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.333+0000 7f4bb64ac640 1 -- 172.21.15.52:0/525271237 shutdown_connections 2024-05-04T12:19:10.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.333+0000 7f4bb64ac640 1 -- 172.21.15.52:0/525271237 wait complete. 2024-05-04T12:19:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bb64ac640 1 Processor -- start 2024-05-04T12:19:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bb64ac640 1 -- start start 2024-05-04T12:19:10.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bb64ac640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb00ccea0 0x7f4bb0134740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bb64ac640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb0145960 0x7f4bb00adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bb64ac640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4bb015c590 0x7f4bb00ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bb64ac640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb006cdc0 con 0x7f4bb00ccea0 2024-05-04T12:19:10.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bb64ac640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb006cc40 con 0x7f4bb0145960 2024-05-04T12:19:10.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bb64ac640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb0145d40 con 0x7f4bb015c590 2024-05-04T12:19:10.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4baffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb00ccea0 0x7f4bb0134740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4baffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb00ccea0 0x7f4bb0134740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56606/0 (socket says 172.21.15.52:56606) 2024-05-04T12:19:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4baffff640 1 -- 172.21.15.52:0/3258942459 learned_addr learned my addr 172.21.15.52:0/3258942459 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4baf7fe640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb0145960 0x7f4bb00adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bb4a49640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4bb015c590 0x7f4bb00ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4baffff640 1 -- 172.21.15.52:0/3258942459 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4bb015c590 msgr2=0x7f4bb00ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4baffff640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4bb015c590 0x7f4bb00ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4baffff640 1 -- 172.21.15.52:0/3258942459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb0145960 msgr2=0x7f4bb00adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4baffff640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb0145960 0x7f4bb00adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4baffff640 1 -- 172.21.15.52:0/3258942459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb0137c30 con 0x7f4bb00ccea0 2024-05-04T12:19:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4baf7fe640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb0145960 0x7f4bb00adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4baffff640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb00ccea0 0x7f4bb0134740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4ba401a0d0 tx=0x7f4ba401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bacff9640 1 -- 172.21.15.52:0/3258942459 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba40bd5f0 con 0x7f4bb00ccea0 2024-05-04T12:19:10.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bacff9640 1 -- 172.21.15.52:0/3258942459 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba40bd790 con 0x7f4bb00ccea0 2024-05-04T12:19:10.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bb64ac640 1 -- 172.21.15.52:0/3258942459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb0137f10 con 0x7f4bb00ccea0 2024-05-04T12:19:10.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bacff9640 1 -- 172.21.15.52:0/3258942459 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba40bdb80 con 0x7f4bb00ccea0 2024-05-04T12:19:10.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bb64ac640 1 -- 172.21.15.52:0/3258942459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb013bff0 con 0x7f4bb00ccea0 2024-05-04T12:19:10.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.337+0000 7f4bb64ac640 1 -- 172.21.15.52:0/3258942459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb0062e50 con 0x7f4bb00ccea0 2024-05-04T12:19:10.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.341+0000 7f4bacff9640 1 -- 172.21.15.52:0/3258942459 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4ba40a9070 con 0x7f4bb00ccea0 2024-05-04T12:19:10.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.341+0000 7f4bacff9640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4b9c0678c0 0x7f4b9c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.341+0000 7f4bacff9640 1 -- 172.21.15.52:0/3258942459 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f4ba4120620 con 0x7f4bb00ccea0 2024-05-04T12:19:10.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.341+0000 7f4baf7fe640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4b9c0678c0 0x7f4b9c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.341+0000 7f4baf7fe640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4b9c0678c0 0x7f4b9c069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4bb01fa870 tx=0x7f4ba0007d70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:10.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.345+0000 7f4bacff9640 1 -- 172.21.15.52:0/3258942459 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4ba40c3060 con 0x7f4bb00ccea0 2024-05-04T12:19:10.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.505+0000 7f4bb64ac640 1 -- 172.21.15.52:0/3258942459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4bb013c410 con 0x7f4bb00ccea0 2024-05-04T12:19:10.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.509+0000 7f4bacff9640 1 -- 172.21.15.52:0/3258942459 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f4ba4016150 con 0x7f4bb00ccea0 2024-05-04T12:19:10.511 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T12:19:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.509+0000 7f4bb64ac640 1 -- 172.21.15.52:0/3258942459 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4b9c0678c0 msgr2=0x7f4b9c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.509+0000 7f4bb64ac640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4b9c0678c0 0x7f4b9c069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4bb01fa870 tx=0x7f4ba0007d70 comp rx=0 tx=0).stop 2024-05-04T12:19:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.509+0000 7f4bb64ac640 1 -- 172.21.15.52:0/3258942459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb00ccea0 msgr2=0x7f4bb0134740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.509+0000 7f4bb64ac640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb00ccea0 0x7f4bb0134740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4ba401a0d0 tx=0x7f4ba401ace0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.509+0000 7f4bb64ac640 1 -- 172.21.15.52:0/3258942459 shutdown_connections 2024-05-04T12:19:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.509+0000 7f4bb64ac640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4b9c0678c0 0x7f4b9c069d80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.509+0000 7f4bb64ac640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4bb015c590 0x7f4bb00ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.509+0000 7f4bb64ac640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb0145960 0x7f4bb00adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.509+0000 7f4bb64ac640 1 --2- 172.21.15.52:0/3258942459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb00ccea0 0x7f4bb0134740 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.509+0000 7f4bb64ac640 1 -- 172.21.15.52:0/3258942459 >> 172.21.15.52:0/3258942459 conn(0x7f4bb0067730 msgr2=0x7f4bb013b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.509+0000 7f4bb64ac640 1 -- 172.21.15.52:0/3258942459 shutdown_connections 2024-05-04T12:19:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.509+0000 7f4bb64ac640 1 -- 172.21.15.52:0/3258942459 wait complete. 2024-05-04T12:19:10.531 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T12:19:10.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.649+0000 7f19eb215640 1 Processor -- start 2024-05-04T12:19:10.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.649+0000 7f19eb215640 1 -- start start 2024-05-04T12:19:10.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19eb215640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19e40cd260 0x7f19e406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19eb215640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19e406d1b0 con 0x7f19e40aa0c0 2024-05-04T12:19:10.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19eb215640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19e406d490 con 0x7f19e415bf50 2024-05-04T12:19:10.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19eb215640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f19e415ca40 con 0x7f19e40cd260 2024-05-04T12:19:10.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e3fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f19e40aa0c0 0x7f19e40ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46878/0 (socket says 172.21.15.52:46878) 2024-05-04T12:19:10.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e3fff640 1 -- 172.21.15.52:0/3587989939 learned_addr learned my addr 172.21.15.52:0/3587989939 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:10.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e97b2640 1 --2- 172.21.15.52:0/3587989939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19e40cd260 0x7f19e406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 -- 172.21.15.52:0/3587989939 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 207905463 0 0) 0x7f19e406d1b0 con 0x7f19e40aa0c0 2024-05-04T12:19:10.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 -- 172.21.15.52:0/3587989939 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19cc003530 con 0x7f19e40aa0c0 2024-05-04T12:19:10.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 -- 172.21.15.52:0/3587989939 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2729239696 0 0) 0x7f19cc003530 con 0x7f19e40aa0c0 2024-05-04T12:19:10.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 -- 172.21.15.52:0/3587989939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19e40cd260 msgr2=0x7f19e406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:10.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 --2- 172.21.15.52:0/3587989939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19e40cd260 0x7f19e406e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 -- 172.21.15.52:0/3587989939 >> v1:172.21.15.111:6789/0 conn(0x7f19e415bf50 legacy=0x7f19e415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 -- 172.21.15.52:0/3587989939 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e406d770 con 0x7f19e40aa0c0 2024-05-04T12:19:10.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 -- 172.21.15.52:0/3587989939 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f19d4001680 con 0x7f19e40aa0c0 2024-05-04T12:19:10.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 -- 172.21.15.52:0/3587989939 >> v1:172.21.15.52:6789/0 conn(0x7f19e40aa0c0 legacy=0x7f19e40ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 --2- 172.21.15.52:0/3587989939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19cc0039f0 0x7f19cc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 --2- 172.21.15.52:0/3587989939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19cc0043c0 0x7f19cc004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 --2- 172.21.15.52:0/3587989939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19e40cd260 0x7f19cc0050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 -- 172.21.15.52:0/3587989939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f19e415ca40 con 0x7f19cc0039f0 2024-05-04T12:19:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 -- 172.21.15.52:0/3587989939 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc00ba30 con 0x7f19cc0043c0 2024-05-04T12:19:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e37fe640 1 -- 172.21.15.52:0/3587989939 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f19cc00bc30 con 0x7f19e40cd260 2024-05-04T12:19:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e8fb1640 1 --2- 172.21.15.52:0/3587989939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19cc0039f0 0x7f19cc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e97b2640 1 --2- 172.21.15.52:0/3587989939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19e40cd260 0x7f19cc0050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e3fff640 1 --2- 172.21.15.52:0/3587989939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19cc0043c0 0x7f19cc004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e8fb1640 1 -- 172.21.15.52:0/3587989939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19e40cd260 msgr2=0x7f19cc0050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e8fb1640 1 --2- 172.21.15.52:0/3587989939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19e40cd260 0x7f19cc0050f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e8fb1640 1 -- 172.21.15.52:0/3587989939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19cc0043c0 msgr2=0x7f19cc004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e8fb1640 1 --2- 172.21.15.52:0/3587989939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19cc0043c0 0x7f19cc004890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.653+0000 7f19e8fb1640 1 -- 172.21.15.52:0/3587989939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19cc003530 con 0x7f19cc0039f0 2024-05-04T12:19:10.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19e8fb1640 1 --2- 172.21.15.52:0/3587989939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19cc0039f0 0x7f19cc003df0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f19d8001860 tx=0x7f19d801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c8c2d4f3bba684 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:10.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19e37fe640 1 -- 172.21.15.52:0/3587989939 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d80b0070 con 0x7f19cc0039f0 2024-05-04T12:19:10.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19e37fe640 1 -- 172.21.15.52:0/3587989939 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d80ab600 con 0x7f19cc0039f0 2024-05-04T12:19:10.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19e37fe640 1 -- 172.21.15.52:0/3587989939 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8001460 con 0x7f19cc0039f0 2024-05-04T12:19:10.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19eb215640 1 -- 172.21.15.52:0/3587989939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19cc0039f0 msgr2=0x7f19cc003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19eb215640 1 --2- 172.21.15.52:0/3587989939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19cc0039f0 0x7f19cc003df0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f19d8001860 tx=0x7f19d801c4f0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19e97b2640 1 -- 172.21.15.52:0/3587989939 reap_dead start 2024-05-04T12:19:10.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19eb215640 1 -- 172.21.15.52:0/3587989939 shutdown_connections 2024-05-04T12:19:10.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19eb215640 1 -- 172.21.15.52:0/3587989939 >> 172.21.15.52:0/3587989939 conn(0x7f19e40b14a0 msgr2=0x7f19e40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:10.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19eb215640 1 -- 172.21.15.52:0/3587989939 shutdown_connections 2024-05-04T12:19:10.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19eb215640 1 -- 172.21.15.52:0/3587989939 wait complete. 2024-05-04T12:19:10.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19eb215640 1 Processor -- start 2024-05-04T12:19:10.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19eb215640 1 -- start start 2024-05-04T12:19:10.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19eb215640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19cc0039f0 0x7f19e413e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19eb215640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19e4070690 0x7f19e41f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19eb215640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e4070b10 0x7f19e41f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19eb215640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f19e406ea60 con 0x7f19cc0039f0 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19eb215640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f19e406cf70 con 0x7f19e4070b10 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19eb215640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f19e406ebe0 con 0x7f19e4070690 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19e8fb1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19cc0039f0 0x7f19e413e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.657+0000 7f19e8fb1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19cc0039f0 0x7f19e413e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56616/0 (socket says 172.21.15.52:56616) 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e8fb1640 1 -- 172.21.15.52:0/1195188000 learned_addr learned my addr 172.21.15.52:0/1195188000 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e3fff640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19e4070690 0x7f19e41f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e97b2640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e4070b10 0x7f19e41f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e8fb1640 1 -- 172.21.15.52:0/1195188000 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19e4070690 msgr2=0x7f19e41f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e8fb1640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19e4070690 0x7f19e41f73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e8fb1640 1 -- 172.21.15.52:0/1195188000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e4070b10 msgr2=0x7f19e41f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e8fb1640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e4070b10 0x7f19e41f78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e8fb1640 1 -- 172.21.15.52:0/1195188000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e40726c0 con 0x7f19cc0039f0 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e3fff640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19e4070690 0x7f19e41f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e97b2640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e4070b10 0x7f19e41f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e8fb1640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19cc0039f0 0x7f19e413e5c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f19d80ae040 tx=0x7f19cc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e17fa640 1 -- 172.21.15.52:0/1195188000 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d80b9020 con 0x7f19cc0039f0 2024-05-04T12:19:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e17fa640 1 -- 172.21.15.52:0/1195188000 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d8001440 con 0x7f19cc0039f0 2024-05-04T12:19:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19eb215640 1 -- 172.21.15.52:0/1195188000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e414f1a0 con 0x7f19cc0039f0 2024-05-04T12:19:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e17fa640 1 -- 172.21.15.52:0/1195188000 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d80b0040 con 0x7f19cc0039f0 2024-05-04T12:19:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19eb215640 1 -- 172.21.15.52:0/1195188000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e414f760 con 0x7f19cc0039f0 2024-05-04T12:19:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19eb215640 1 -- 172.21.15.52:0/1195188000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e4062e50 con 0x7f19cc0039f0 2024-05-04T12:19:10.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e17fa640 1 -- 172.21.15.52:0/1195188000 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f19d80bc020 con 0x7f19cc0039f0 2024-05-04T12:19:10.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e17fa640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19d00678f0 0x7f19d0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:10.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e17fa640 1 -- 172.21.15.52:0/1195188000 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f19d811f990 con 0x7f19cc0039f0 2024-05-04T12:19:10.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e3fff640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19d00678f0 0x7f19d0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:10.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.661+0000 7f19e3fff640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19d00678f0 0x7f19d0069db0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f19d4001ec0 tx=0x7f19d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:10.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.665+0000 7f19e17fa640 1 -- 172.21.15.52:0/1195188000 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f19d80c3030 con 0x7f19cc0039f0 2024-05-04T12:19:10.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.865+0000 7f19eb215640 1 -- 172.21.15.52:0/1195188000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19e4075320 con 0x7f19cc0039f0 2024-05-04T12:19:10.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.865+0000 7f19e17fa640 1 -- 172.21.15.52:0/1195188000 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7f19e40726c0 con 0x7f19cc0039f0 2024-05-04T12:19:10.869 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:10.869 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6836/482177111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":482177111},{"type":"v1","addr":"172.21.15.52:6836","nonce":482177111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/163453709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6835","nonce":163453709},{"type":"v1","addr":"172.21.15.52:6837","nonce":163453709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3596668218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3596668218},{"type":"v1","addr":"172.21.15.111:6833","nonce":3596668218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-04T12:18:22.189525+0000","modified":"2024-05-04T12:18:25.240402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/1656435333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1656435333},{"type":"v1","addr":"172.21.15.111:6835","nonce":1656435333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-05-04T12:19:10.869 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 7 2024-05-04T12:19:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.869+0000 7f19eb215640 1 -- 172.21.15.52:0/1195188000 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19d00678f0 msgr2=0x7f19d0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.869+0000 7f19eb215640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19d00678f0 0x7f19d0069db0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f19d4001ec0 tx=0x7f19d4009040 comp rx=0 tx=0).stop 2024-05-04T12:19:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.869+0000 7f19eb215640 1 -- 172.21.15.52:0/1195188000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19cc0039f0 msgr2=0x7f19e413e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.869+0000 7f19eb215640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19cc0039f0 0x7f19e413e5c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f19d80ae040 tx=0x7f19cc005840 comp rx=0 tx=0).stop 2024-05-04T12:19:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.869+0000 7f19eb215640 1 -- 172.21.15.52:0/1195188000 shutdown_connections 2024-05-04T12:19:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.869+0000 7f19eb215640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19d00678f0 0x7f19d0069db0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.869+0000 7f19eb215640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e4070b10 0x7f19e41f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.869+0000 7f19eb215640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19e4070690 0x7f19e41f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.869+0000 7f19eb215640 1 --2- 172.21.15.52:0/1195188000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19cc0039f0 0x7f19e413e5c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.869+0000 7f19eb215640 1 -- 172.21.15.52:0/1195188000 >> 172.21.15.52:0/1195188000 conn(0x7f19e40b14a0 msgr2=0x7f19e40a9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.869+0000 7f19eb215640 1 -- 172.21.15.52:0/1195188000 shutdown_connections 2024-05-04T12:19:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:10.869+0000 7f19eb215640 1 -- 172.21.15.52:0/1195188000 wait complete. 2024-05-04T12:19:10.889 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:19:11.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.001+0000 7fdcce787640 1 Processor -- start 2024-05-04T12:19:11.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.001+0000 7fdcce787640 1 -- start start 2024-05-04T12:19:11.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcce787640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdcc80ccf10 0x7fdcc80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcce787640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdcc80d28c0 0x7fdcc80d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcce787640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc80cfec0 con 0x7fdcc815c450 2024-05-04T12:19:11.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcce787640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc8160af0 con 0x7fdcc80d28c0 2024-05-04T12:19:11.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcce787640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc8160dd0 con 0x7fdcc80ccf10 2024-05-04T12:19:11.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc7fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fdcc815c450 0x7fdcc80ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46884/0 (socket says 172.21.15.52:46884) 2024-05-04T12:19:11.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc7fff640 1 -- 172.21.15.52:0/501891873 learned_addr learned my addr 172.21.15.52:0/501891873 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:11.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 -- 172.21.15.52:0/501891873 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3933341899 0 0) 0x7fdcc80cfec0 con 0x7fdcc815c450 2024-05-04T12:19:11.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcbf7fe640 1 --2- 172.21.15.52:0/501891873 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdcc80ccf10 0x7fdcc80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcccd24640 1 --2- 172.21.15.52:0/501891873 >> v2:172.21.15.111:3300/0 conn(0x7fdcc80d28c0 0x7fdcc80d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 -- 172.21.15.52:0/501891873 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcb4003530 con 0x7fdcc815c450 2024-05-04T12:19:11.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 -- 172.21.15.52:0/501891873 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3330722901 0 0) 0x7fdcb4003530 con 0x7fdcc815c450 2024-05-04T12:19:11.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 -- 172.21.15.52:0/501891873 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdcc80ccf10 msgr2=0x7fdcc80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:11.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 --2- 172.21.15.52:0/501891873 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdcc80ccf10 0x7fdcc80d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 -- 172.21.15.52:0/501891873 >> v2:172.21.15.111:3300/0 conn(0x7fdcc80d28c0 msgr2=0x7fdcc80d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:11.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 --2- 172.21.15.52:0/501891873 >> v2:172.21.15.111:3300/0 conn(0x7fdcc80d28c0 0x7fdcc80d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 -- 172.21.15.52:0/501891873 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc80d1c80 con 0x7fdcc815c450 2024-05-04T12:19:11.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 -- 172.21.15.52:0/501891873 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7fdcb8001680 con 0x7fdcc815c450 2024-05-04T12:19:11.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 -- 172.21.15.52:0/501891873 >> v1:172.21.15.52:6789/0 conn(0x7fdcc815c450 legacy=0x7fdcc80ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 --2- 172.21.15.52:0/501891873 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdcb40039c0 0x7fdcb4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 --2- 172.21.15.52:0/501891873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcb4004320 0x7fdcb4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 --2- 172.21.15.52:0/501891873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcb4004e60 0x7fdcb40052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 -- 172.21.15.52:0/501891873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8160af0 con 0x7fdcb4004320 2024-05-04T12:19:11.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 -- 172.21.15.52:0/501891873 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8160dd0 con 0x7fdcb4004e60 2024-05-04T12:19:11.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcc77fe640 1 -- 172.21.15.52:0/501891873 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fdcb400bd50 con 0x7fdcb40039c0 2024-05-04T12:19:11.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcbf7fe640 1 --2- 172.21.15.52:0/501891873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcb4004320 0x7fdcb4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcccd24640 1 --2- 172.21.15.52:0/501891873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcb4004e60 0x7fdcb40052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcbf7fe640 1 -- 172.21.15.52:0/501891873 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdcb40039c0 msgr2=0x7fdcb4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:11.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcbf7fe640 1 --2- 172.21.15.52:0/501891873 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdcb40039c0 0x7fdcb4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcbf7fe640 1 -- 172.21.15.52:0/501891873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcb4004e60 msgr2=0x7fdcb40052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:11.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcbf7fe640 1 --2- 172.21.15.52:0/501891873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcb4004e60 0x7fdcb40052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.005+0000 7fdcbf7fe640 1 -- 172.21.15.52:0/501891873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb4003530 con 0x7fdcb4004320 2024-05-04T12:19:11.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.009+0000 7fdcbf7fe640 1 --2- 172.21.15.52:0/501891873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcb4004320 0x7fdcb4004810 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009ab0 tx=0x7fdcb001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4877b7ee35f2a9d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:11.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.009+0000 7fdcc77fe640 1 -- 172.21.15.52:0/501891873 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00b2070 con 0x7fdcb4004320 2024-05-04T12:19:11.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.009+0000 7fdcc77fe640 1 -- 172.21.15.52:0/501891873 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb00a5680 con 0x7fdcb4004320 2024-05-04T12:19:11.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.009+0000 7fdcc77fe640 1 -- 172.21.15.52:0/501891873 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00096b0 con 0x7fdcb4004320 2024-05-04T12:19:11.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.009+0000 7fdcce787640 1 -- 172.21.15.52:0/501891873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcb4004320 msgr2=0x7fdcb4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.009+0000 7fdcce787640 1 --2- 172.21.15.52:0/501891873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcb4004320 0x7fdcb4004810 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009ab0 tx=0x7fdcb001ca90 comp rx=0 tx=0).stop 2024-05-04T12:19:11.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.009+0000 7fdcccd24640 1 -- 172.21.15.52:0/501891873 reap_dead start 2024-05-04T12:19:11.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.009+0000 7fdcce787640 1 -- 172.21.15.52:0/501891873 shutdown_connections 2024-05-04T12:19:11.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.009+0000 7fdcce787640 1 -- 172.21.15.52:0/501891873 >> 172.21.15.52:0/501891873 conn(0x7fdcc806b210 msgr2=0x7fdcc806b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:11.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.009+0000 7fdcce787640 1 -- 172.21.15.52:0/501891873 shutdown_connections 2024-05-04T12:19:11.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.009+0000 7fdcce787640 1 -- 172.21.15.52:0/501891873 wait complete. 2024-05-04T12:19:11.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.009+0000 7fdcce787640 1 Processor -- start 2024-05-04T12:19:11.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.009+0000 7fdcce787640 1 -- start start 2024-05-04T12:19:11.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcce787640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdcc8159000 0x7fdcc8144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcce787640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcc8158380 0x7fdcc8142a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcce787640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcc815f4c0 0x7fdcc815f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcce787640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc815c9b0 con 0x7fdcc8158380 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcce787640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc815c830 con 0x7fdcc815f4c0 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcce787640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc80d2380 con 0x7fdcc8159000 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcbf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcc8158380 0x7fdcc8142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcbf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcc8158380 0x7fdcc8142a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56636/0 (socket says 172.21.15.52:56636) 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcbf7fe640 1 -- 172.21.15.52:0/4019234186 learned_addr learned my addr 172.21.15.52:0/4019234186 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcc7fff640 1 --2- 172.21.15.52:0/4019234186 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdcc8159000 0x7fdcc8144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcccd24640 1 --2- 172.21.15.52:0/4019234186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcc815f4c0 0x7fdcc815f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcbf7fe640 1 -- 172.21.15.52:0/4019234186 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdcc8159000 msgr2=0x7fdcc8144490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcbf7fe640 1 --2- 172.21.15.52:0/4019234186 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdcc8159000 0x7fdcc8144490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcbf7fe640 1 -- 172.21.15.52:0/4019234186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcc815f4c0 msgr2=0x7fdcc815f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcbf7fe640 1 --2- 172.21.15.52:0/4019234186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcc815f4c0 0x7fdcc815f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcbf7fe640 1 -- 172.21.15.52:0/4019234186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc81433f0 con 0x7fdcc8158380 2024-05-04T12:19:11.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcbf7fe640 1 --2- 172.21.15.52:0/4019234186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcc8158380 0x7fdcc8142a80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00b0040 tx=0x7fdcb0019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:11.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcc57fa640 1 -- 172.21.15.52:0/4019234186 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00b2040 con 0x7fdcc8158380 2024-05-04T12:19:11.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcc57fa640 1 -- 172.21.15.52:0/4019234186 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb00a5e10 con 0x7fdcc8158380 2024-05-04T12:19:11.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcce787640 1 -- 172.21.15.52:0/4019234186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc81436d0 con 0x7fdcc8158380 2024-05-04T12:19:11.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcc57fa640 1 -- 172.21.15.52:0/4019234186 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0004260 con 0x7fdcc8158380 2024-05-04T12:19:11.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcce787640 1 -- 172.21.15.52:0/4019234186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc8143c90 con 0x7fdcc8158380 2024-05-04T12:19:11.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcce787640 1 -- 172.21.15.52:0/4019234186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc8062e50 con 0x7fdcc8158380 2024-05-04T12:19:11.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcc57fa640 1 -- 172.21.15.52:0/4019234186 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdcb00bf020 con 0x7fdcc8158380 2024-05-04T12:19:11.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcc57fa640 1 --2- 172.21.15.52:0/4019234186 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdca80678c0 0x7fdca8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcc57fa640 1 -- 172.21.15.52:0/4019234186 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7fdcb011f7f0 con 0x7fdcc8158380 2024-05-04T12:19:11.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.013+0000 7fdcc7fff640 1 --2- 172.21.15.52:0/4019234186 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdca80678c0 0x7fdca8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.017+0000 7fdcc7fff640 1 --2- 172.21.15.52:0/4019234186 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdca80678c0 0x7fdca8069d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8001000 tx=0x7fdcb8001500 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:11.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.017+0000 7fdcc57fa640 1 -- 172.21.15.52:0/4019234186 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdcb00167a0 con 0x7fdcc8158380 2024-05-04T12:19:11.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.177+0000 7fdcce787640 1 -- 172.21.15.52:0/4019234186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdcc80cf270 con 0x7fdcc8158380 2024-05-04T12:19:11.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.181+0000 7fdcc57fa640 1 -- 172.21.15.52:0/4019234186 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16913 (secure 0 0 0) 0x7fdcb00f0380 con 0x7fdcc8158380 2024-05-04T12:19:11.184 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:11.184 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:19:04.046290+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:19:11.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.181+0000 7fdcce787640 1 -- 172.21.15.52:0/4019234186 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdca80678c0 msgr2=0x7fdca8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.181+0000 7fdcce787640 1 --2- 172.21.15.52:0/4019234186 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdca80678c0 0x7fdca8069d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8001000 tx=0x7fdcb8001500 comp rx=0 tx=0).stop 2024-05-04T12:19:11.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.181+0000 7fdcce787640 1 -- 172.21.15.52:0/4019234186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcc8158380 msgr2=0x7fdcc8142a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.181+0000 7fdcce787640 1 --2- 172.21.15.52:0/4019234186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcc8158380 0x7fdcc8142a80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00b0040 tx=0x7fdcb0019830 comp rx=0 tx=0).stop 2024-05-04T12:19:11.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.181+0000 7fdcce787640 1 -- 172.21.15.52:0/4019234186 shutdown_connections 2024-05-04T12:19:11.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.181+0000 7fdcce787640 1 --2- 172.21.15.52:0/4019234186 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdca80678c0 0x7fdca8069d80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.181+0000 7fdcce787640 1 --2- 172.21.15.52:0/4019234186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcc815f4c0 0x7fdcc815f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.181+0000 7fdcce787640 1 --2- 172.21.15.52:0/4019234186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcc8158380 0x7fdcc8142a80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.181+0000 7fdcce787640 1 --2- 172.21.15.52:0/4019234186 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdcc8159000 0x7fdcc8144490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.181+0000 7fdcce787640 1 -- 172.21.15.52:0/4019234186 >> 172.21.15.52:0/4019234186 conn(0x7fdcc806b210 msgr2=0x7fdcc8070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:11.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.181+0000 7fdcce787640 1 -- 172.21.15.52:0/4019234186 shutdown_connections 2024-05-04T12:19:11.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.181+0000 7fdcce787640 1 -- 172.21.15.52:0/4019234186 wait complete. 2024-05-04T12:19:11.202 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-04T12:19:11.203 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T12:19:11.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb3fdf640 1 Processor -- start 2024-05-04T12:19:11.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb3fdf640 1 -- start start 2024-05-04T12:19:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb3fdf640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdeac15a530 0x7fdeac15e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb3fdf640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdeac15a910 0x7fdeac071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb3fdf640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdeac15ea30 con 0x7fdeac0cc2c0 2024-05-04T12:19:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb3fdf640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fdeac06f080 con 0x7fdeac15a530 2024-05-04T12:19:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb3fdf640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fdeac15ed10 con 0x7fdeac15a910 2024-05-04T12:19:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb1d7b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdeac15a530 0x7fdeac15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb1d7b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdeac15a530 0x7fdeac15e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56650/0 (socket says 172.21.15.52:56650) 2024-05-04T12:19:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb1d7b640 1 -- 172.21.15.52:0/886415273 learned_addr learned my addr 172.21.15.52:0/886415273 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb157a640 1 --2- 172.21.15.52:0/886415273 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdeac15a910 0x7fdeac071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb1d7b640 1 -- 172.21.15.52:0/886415273 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdeac15a910 msgr2=0x7fdeac071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb1d7b640 1 --2- 172.21.15.52:0/886415273 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdeac15a910 0x7fdeac071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb1d7b640 1 -- 172.21.15.52:0/886415273 >> v1:172.21.15.111:6789/0 conn(0x7fdeac0cc2c0 legacy=0x7fdeac070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb1d7b640 1 -- 172.21.15.52:0/886415273 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeac15eff0 con 0x7fdeac15a530 2024-05-04T12:19:11.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.321+0000 7fdeb1d7b640 1 --2- 172.21.15.52:0/886415273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeac15a530 0x7fdeac15e330 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009e50 tx=0x7fdea801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47439a6631c480b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:11.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb0d79640 1 -- 172.21.15.52:0/886415273 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea80a5780 con 0x7fdeac15a530 2024-05-04T12:19:11.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb0d79640 1 -- 172.21.15.52:0/886415273 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea80a5920 con 0x7fdeac15a530 2024-05-04T12:19:11.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb0d79640 1 -- 172.21.15.52:0/886415273 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea80a5cf0 con 0x7fdeac15a530 2024-05-04T12:19:11.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/886415273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeac15a530 msgr2=0x7fdeac15e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 --2- 172.21.15.52:0/886415273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeac15a530 0x7fdeac15e330 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009e50 tx=0x7fdea801c9f0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/886415273 shutdown_connections 2024-05-04T12:19:11.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 --2- 172.21.15.52:0/886415273 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdeac15a910 0x7fdeac071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 --2- 172.21.15.52:0/886415273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeac15a530 0x7fdeac15e330 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/886415273 >> 172.21.15.52:0/886415273 conn(0x7fdeac0b14a0 msgr2=0x7fdeac0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:11.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/886415273 shutdown_connections 2024-05-04T12:19:11.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/886415273 wait complete. 2024-05-04T12:19:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 Processor -- start 2024-05-04T12:19:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 -- start start 2024-05-04T12:19:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdeac0cc2c0 0x7fdeac13ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeac15a530 0x7fdeac0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdeac15a910 0x7fdeac151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac0ccf60 con 0x7fdeac15a530 2024-05-04T12:19:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac0ccde0 con 0x7fdeac0cc2c0 2024-05-04T12:19:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fdeac0cd0e0 con 0x7fdeac15a910 2024-05-04T12:19:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb157a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeac15a530 0x7fdeac0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb157a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeac15a530 0x7fdeac0b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56654/0 (socket says 172.21.15.52:56654) 2024-05-04T12:19:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb157a640 1 -- 172.21.15.52:0/329242409 learned_addr learned my addr 172.21.15.52:0/329242409 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb1d7b640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdeac0cc2c0 0x7fdeac13ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb257c640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdeac15a910 0x7fdeac151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb157a640 1 -- 172.21.15.52:0/329242409 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdeac15a910 msgr2=0x7fdeac151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb157a640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdeac15a910 0x7fdeac151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb157a640 1 -- 172.21.15.52:0/329242409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdeac0cc2c0 msgr2=0x7fdeac13ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb157a640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdeac0cc2c0 0x7fdeac13ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb157a640 1 -- 172.21.15.52:0/329242409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeac13c4b0 con 0x7fdeac15a530 2024-05-04T12:19:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb257c640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdeac15a910 0x7fdeac151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb157a640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeac15a530 0x7fdeac0b0160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00c5e0 tx=0x7fde9c00caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdea27fc640 1 -- 172.21.15.52:0/329242409 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c015070 con 0x7fdeac15a530 2024-05-04T12:19:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdea27fc640 1 -- 172.21.15.52:0/329242409 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde9c004220 con 0x7fdeac15a530 2024-05-04T12:19:11.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/329242409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeac13b6f0 con 0x7fdeac15a530 2024-05-04T12:19:11.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdea27fc640 1 -- 172.21.15.52:0/329242409 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c003620 con 0x7fdeac15a530 2024-05-04T12:19:11.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.325+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/329242409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeac13bcb0 con 0x7fdeac15a530 2024-05-04T12:19:11.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.329+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/329242409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeac062e50 con 0x7fdeac15a530 2024-05-04T12:19:11.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.329+0000 7fdea27fc640 1 -- 172.21.15.52:0/329242409 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fde9c0037e0 con 0x7fdeac15a530 2024-05-04T12:19:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.329+0000 7fdea27fc640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fde900678c0 0x7fde90069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.329+0000 7fdeb1d7b640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fde900678c0 0x7fde90069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.329+0000 7fdea27fc640 1 -- 172.21.15.52:0/329242409 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7fde9c085b80 con 0x7fdeac15a530 2024-05-04T12:19:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.329+0000 7fdeb1d7b640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fde900678c0 0x7fde90069d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdea80ae000 tx=0x7fdea80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:11.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.333+0000 7fdea27fc640 1 -- 172.21.15.52:0/329242409 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fde9c056590 con 0x7fdeac15a530 2024-05-04T12:19:11.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.521+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/329242409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdeac075320 con 0x7fdeac15a530 2024-05-04T12:19:11.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.525+0000 7fdea27fc640 1 -- 172.21.15.52:0/329242409 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7fde9c055e90 con 0x7fdeac15a530 2024-05-04T12:19:11.527 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:11.527 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6836/482177111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":482177111},{"type":"v1","addr":"172.21.15.52:6836","nonce":482177111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/163453709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6835","nonce":163453709},{"type":"v1","addr":"172.21.15.52:6837","nonce":163453709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3596668218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3596668218},{"type":"v1","addr":"172.21.15.111:6833","nonce":3596668218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-04T12:18:22.189525+0000","modified":"2024-05-04T12:18:25.240402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/1656435333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1656435333},{"type":"v1","addr":"172.21.15.111:6835","nonce":1656435333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-05-04T12:19:11.527 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 7 2024-05-04T12:19:11.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.525+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/329242409 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fde900678c0 msgr2=0x7fde90069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.525+0000 7fdeb3fdf640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fde900678c0 0x7fde90069d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdea80ae000 tx=0x7fdea80a7070 comp rx=0 tx=0).stop 2024-05-04T12:19:11.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.525+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/329242409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeac15a530 msgr2=0x7fdeac0b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.525+0000 7fdeb3fdf640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeac15a530 0x7fdeac0b0160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00c5e0 tx=0x7fde9c00caa0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.525+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/329242409 shutdown_connections 2024-05-04T12:19:11.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.525+0000 7fdeb3fdf640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fde900678c0 0x7fde90069d80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.525+0000 7fdeb3fdf640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdeac15a910 0x7fdeac151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.525+0000 7fdeb3fdf640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeac15a530 0x7fdeac0b0160 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.525+0000 7fdeb3fdf640 1 --2- 172.21.15.52:0/329242409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdeac0cc2c0 0x7fdeac13ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.525+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/329242409 >> 172.21.15.52:0/329242409 conn(0x7fdeac0b14a0 msgr2=0x7fdeac0cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:11.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.525+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/329242409 shutdown_connections 2024-05-04T12:19:11.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.525+0000 7fdeb3fdf640 1 -- 172.21.15.52:0/329242409 wait complete. 2024-05-04T12:19:11.545 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T12:19:11.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc32e2640 1 Processor -- start 2024-05-04T12:19:11.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc32e2640 1 -- start start 2024-05-04T12:19:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc32e2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fafbc0aac30 0x7fafbc0cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc32e2640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafbc071f30 con 0x7fafbc158810 2024-05-04T12:19:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc32e2640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafbc144030 con 0x7fafbc143b50 2024-05-04T12:19:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc32e2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fafbc0ab740 con 0x7fafbc0aac30 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc107e640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fafbc158810 0x7fafbc0aa520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46898/0 (socket says 172.21.15.52:46898) 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc107e640 1 -- 172.21.15.52:0/3758905681 learned_addr learned my addr 172.21.15.52:0/3758905681 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc087d640 1 --2- 172.21.15.52:0/3758905681 >> v2:172.21.15.111:3300/0 conn(0x7fafbc0aac30 0x7fafbc0cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 -- 172.21.15.52:0/3758905681 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 494106004 0 0) 0x7fafbc071f30 con 0x7fafbc158810 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 -- 172.21.15.52:0/3758905681 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafa8003530 con 0x7fafbc158810 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 -- 172.21.15.52:0/3758905681 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1519230330 0 0) 0x7fafa8003530 con 0x7fafbc158810 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 -- 172.21.15.52:0/3758905681 >> v2:172.21.15.111:3300/0 conn(0x7fafbc0aac30 msgr2=0x7fafbc0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 --2- 172.21.15.52:0/3758905681 >> v2:172.21.15.111:3300/0 conn(0x7fafbc0aac30 0x7fafbc0cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 -- 172.21.15.52:0/3758905681 >> v1:172.21.15.111:6789/0 conn(0x7fafbc143b50 legacy=0x7fafbc0abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 -- 172.21.15.52:0/3758905681 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc144310 con 0x7fafbc158810 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 -- 172.21.15.52:0/3758905681 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7fafb80016a0 con 0x7fafbc158810 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 -- 172.21.15.52:0/3758905681 >> v1:172.21.15.52:6789/0 conn(0x7fafbc158810 legacy=0x7fafbc0aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 --2- 172.21.15.52:0/3758905681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa80039c0 0x7fafa8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 --2- 172.21.15.52:0/3758905681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafa8004810 0x7fafa8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 --2- 172.21.15.52:0/3758905681 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fafa80051d0 0x7fafa800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 -- 172.21.15.52:0/3758905681 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc0ab740 con 0x7fafa8004810 2024-05-04T12:19:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 -- 172.21.15.52:0/3758905681 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fafa800c070 con 0x7fafa80039c0 2024-05-04T12:19:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafb3fff640 1 -- 172.21.15.52:0/3758905681 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fafa800c270 con 0x7fafa80051d0 2024-05-04T12:19:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc087d640 1 --2- 172.21.15.52:0/3758905681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafa8004810 0x7fafa8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc107e640 1 --2- 172.21.15.52:0/3758905681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa80039c0 0x7fafa8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc087d640 1 -- 172.21.15.52:0/3758905681 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fafa80051d0 msgr2=0x7fafa800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc187f640 1 --2- 172.21.15.52:0/3758905681 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fafa80051d0 0x7fafa800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc087d640 1 --2- 172.21.15.52:0/3758905681 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fafa80051d0 0x7fafa800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc087d640 1 -- 172.21.15.52:0/3758905681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa80039c0 msgr2=0x7fafa8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc087d640 1 --2- 172.21.15.52:0/3758905681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa80039c0 0x7fafa8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc087d640 1 -- 172.21.15.52:0/3758905681 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa8003530 con 0x7fafa8004810 2024-05-04T12:19:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc187f640 1 --2- 172.21.15.52:0/3758905681 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fafa80051d0 0x7fafa800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.665+0000 7fafc187f640 1 -- 172.21.15.52:0/3758905681 reap_dead start 2024-05-04T12:19:11.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.669+0000 7fafc087d640 1 --2- 172.21.15.52:0/3758905681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafa8004810 0x7fafa8004c90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fafac009ac0 tx=0x7fafac01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9baab3f759a6eae server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:11.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.669+0000 7fafb3fff640 1 -- 172.21.15.52:0/3758905681 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac0b2070 con 0x7fafa8004810 2024-05-04T12:19:11.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.669+0000 7fafb3fff640 1 -- 172.21.15.52:0/3758905681 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafac0a5530 con 0x7fafa8004810 2024-05-04T12:19:11.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.669+0000 7fafb3fff640 1 -- 172.21.15.52:0/3758905681 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac009690 con 0x7fafa8004810 2024-05-04T12:19:11.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.669+0000 7fafc32e2640 1 -- 172.21.15.52:0/3758905681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafa8004810 msgr2=0x7fafa8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.669+0000 7fafc32e2640 1 --2- 172.21.15.52:0/3758905681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafa8004810 0x7fafa8004c90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fafac009ac0 tx=0x7fafac01ca50 comp rx=0 tx=0).stop 2024-05-04T12:19:11.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.669+0000 7fafc32e2640 1 -- 172.21.15.52:0/3758905681 shutdown_connections 2024-05-04T12:19:11.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.669+0000 7fafc32e2640 1 --2- 172.21.15.52:0/3758905681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafa8004810 0x7fafa8004c90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.669+0000 7fafc32e2640 1 -- 172.21.15.52:0/3758905681 >> 172.21.15.52:0/3758905681 conn(0x7fafbc067070 msgr2=0x7fafbc067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:11.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.669+0000 7fafc32e2640 1 -- 172.21.15.52:0/3758905681 shutdown_connections 2024-05-04T12:19:11.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.669+0000 7fafc32e2640 1 -- 172.21.15.52:0/3758905681 wait complete. 2024-05-04T12:19:11.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.669+0000 7fafc32e2640 1 Processor -- start 2024-05-04T12:19:11.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.669+0000 7fafc32e2640 1 -- start start 2024-05-04T12:19:11.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc32e2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa8004810 0x7fafbc1ea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc32e2640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fafbc1f7920 0x7fafbc13d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc32e2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafbc1f7da0 0x7fafbc13d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc32e2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc158bf0 con 0x7fafbc1f7da0 2024-05-04T12:19:11.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc32e2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1448d0 con 0x7fafa8004810 2024-05-04T12:19:11.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc32e2640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc158d70 con 0x7fafbc1f7920 2024-05-04T12:19:11.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc187f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafbc1f7da0 0x7fafbc13d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc187f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafbc1f7da0 0x7fafbc13d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56658/0 (socket says 172.21.15.52:56658) 2024-05-04T12:19:11.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc187f640 1 -- 172.21.15.52:0/1843971103 learned_addr learned my addr 172.21.15.52:0/1843971103 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:11.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc107e640 1 --2- 172.21.15.52:0/1843971103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa8004810 0x7fafbc1ea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc087d640 1 --2- 172.21.15.52:0/1843971103 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fafbc1f7920 0x7fafbc13d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc187f640 1 -- 172.21.15.52:0/1843971103 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fafbc1f7920 msgr2=0x7fafbc13d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc187f640 1 --2- 172.21.15.52:0/1843971103 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fafbc1f7920 0x7fafbc13d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc187f640 1 -- 172.21.15.52:0/1843971103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa8004810 msgr2=0x7fafbc1ea1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc187f640 1 --2- 172.21.15.52:0/1843971103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa8004810 0x7fafbc1ea1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc187f640 1 -- 172.21.15.52:0/1843971103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc1ea720 con 0x7fafbc1f7da0 2024-05-04T12:19:11.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc187f640 1 --2- 172.21.15.52:0/1843971103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafbc1f7da0 0x7fafbc13d870 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fafbc143f60 tx=0x7fafb400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:11.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafb1ffb640 1 -- 172.21.15.52:0/1843971103 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4018070 con 0x7fafbc1f7da0 2024-05-04T12:19:11.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafb1ffb640 1 -- 172.21.15.52:0/1843971103 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb4003db0 con 0x7fafbc1f7da0 2024-05-04T12:19:11.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc32e2640 1 -- 172.21.15.52:0/1843971103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc1e89a0 con 0x7fafbc1f7da0 2024-05-04T12:19:11.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafb1ffb640 1 -- 172.21.15.52:0/1843971103 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb40052b0 con 0x7fafbc1f7da0 2024-05-04T12:19:11.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc32e2640 1 -- 172.21.15.52:0/1843971103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc1e8f60 con 0x7fafbc1f7da0 2024-05-04T12:19:11.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc32e2640 1 -- 172.21.15.52:0/1843971103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafbc062e50 con 0x7fafbc1f7da0 2024-05-04T12:19:11.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafb1ffb640 1 -- 172.21.15.52:0/1843971103 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fafb401b070 con 0x7fafbc1f7da0 2024-05-04T12:19:11.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafb1ffb640 1 --2- 172.21.15.52:0/1843971103 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fafa00678c0 0x7fafa0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:11.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafb1ffb640 1 -- 172.21.15.52:0/1843971103 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7fafb4085360 con 0x7fafbc1f7da0 2024-05-04T12:19:11.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc107e640 1 --2- 172.21.15.52:0/1843971103 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fafa00678c0 0x7fafa0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:11.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.673+0000 7fafc107e640 1 --2- 172.21.15.52:0/1843971103 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fafa00678c0 0x7fafa0069d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fafb8001cf0 tx=0x7fafb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:11.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.677+0000 7fafb1ffb640 1 -- 172.21.15.52:0/1843971103 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fafb4055e10 con 0x7fafbc1f7da0 2024-05-04T12:19:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.873+0000 7fafc32e2640 1 -- 172.21.15.52:0/1843971103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafbc075320 con 0x7fafbc1f7da0 2024-05-04T12:19:11.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.873+0000 7fafb1ffb640 1 -- 172.21.15.52:0/1843971103 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7fafb4055710 con 0x7fafbc1f7da0 2024-05-04T12:19:11.878 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:11.878 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6836/482177111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":482177111},{"type":"v1","addr":"172.21.15.52:6836","nonce":482177111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/163453709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6835","nonce":163453709},{"type":"v1","addr":"172.21.15.52:6837","nonce":163453709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3596668218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3596668218},{"type":"v1","addr":"172.21.15.111:6833","nonce":3596668218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-04T12:18:22.189525+0000","modified":"2024-05-04T12:18:25.240402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/1656435333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1656435333},{"type":"v1","addr":"172.21.15.111:6835","nonce":1656435333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-05-04T12:19:11.878 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 7 2024-05-04T12:19:11.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.877+0000 7fafc32e2640 1 -- 172.21.15.52:0/1843971103 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fafa00678c0 msgr2=0x7fafa0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.877+0000 7fafc32e2640 1 --2- 172.21.15.52:0/1843971103 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fafa00678c0 0x7fafa0069d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fafb8001cf0 tx=0x7fafb8009040 comp rx=0 tx=0).stop 2024-05-04T12:19:11.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.877+0000 7fafc32e2640 1 -- 172.21.15.52:0/1843971103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafbc1f7da0 msgr2=0x7fafbc13d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:11.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.877+0000 7fafc32e2640 1 --2- 172.21.15.52:0/1843971103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafbc1f7da0 0x7fafbc13d870 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fafbc143f60 tx=0x7fafb400d9d0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.877+0000 7fafc32e2640 1 -- 172.21.15.52:0/1843971103 shutdown_connections 2024-05-04T12:19:11.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.877+0000 7fafc32e2640 1 --2- 172.21.15.52:0/1843971103 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fafa00678c0 0x7fafa0069d80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.877+0000 7fafc32e2640 1 --2- 172.21.15.52:0/1843971103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafbc1f7da0 0x7fafbc13d870 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.877+0000 7fafc32e2640 1 --2- 172.21.15.52:0/1843971103 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fafbc1f7920 0x7fafbc13d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.877+0000 7fafc32e2640 1 --2- 172.21.15.52:0/1843971103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa8004810 0x7fafbc1ea1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:11.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.877+0000 7fafc32e2640 1 -- 172.21.15.52:0/1843971103 >> 172.21.15.52:0/1843971103 conn(0x7fafbc067070 msgr2=0x7fafbc0d3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:11.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.877+0000 7fafc32e2640 1 -- 172.21.15.52:0/1843971103 shutdown_connections 2024-05-04T12:19:11.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:11.877+0000 7fafc32e2640 1 -- 172.21.15.52:0/1843971103 wait complete. 2024-05-04T12:19:11.898 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:19:12.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f12018ba640 1 Processor -- start 2024-05-04T12:19:12.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f12018ba640 1 -- start start 2024-05-04T12:19:12.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f12018ba640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f11fc15bf50 0x7f11fc15c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f12018ba640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f11fc0aa0c0 0x7f11fc0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f12018ba640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11fc06d1b0 con 0x7f11fc0aa550 2024-05-04T12:19:12.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f12018ba640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f11fc06d490 con 0x7f11fc15bf50 2024-05-04T12:19:12.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f12018ba640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f11fc06d770 con 0x7f11fc0aa0c0 2024-05-04T12:19:12.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f11faffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f11fc15bf50 0x7f11fc15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f11fb7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f11fc0aa550 0x7f11fc06e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46904/0 (socket says 172.21.15.52:46904) 2024-05-04T12:19:12.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f11fb7fe640 1 -- 172.21.15.52:0/1176740260 learned_addr learned my addr 172.21.15.52:0/1176740260 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:12.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f11fa7fc640 1 --2- 172.21.15.52:0/1176740260 >> v2:172.21.15.111:3300/0 conn(0x7f11fc0aa0c0 0x7f11fc0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f11faffd640 1 -- 172.21.15.52:0/1176740260 >> v2:172.21.15.111:3300/0 conn(0x7f11fc0aa0c0 msgr2=0x7f11fc0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:12.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f11faffd640 1 --2- 172.21.15.52:0/1176740260 >> v2:172.21.15.111:3300/0 conn(0x7f11fc0aa0c0 0x7f11fc0ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f11faffd640 1 -- 172.21.15.52:0/1176740260 >> v1:172.21.15.52:6789/0 conn(0x7f11fc0aa550 legacy=0x7f11fc06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f11faffd640 1 -- 172.21.15.52:0/1176740260 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc06da50 con 0x7f11fc15bf50 2024-05-04T12:19:12.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.021+0000 7f11f9ffb640 1 -- 172.21.15.52:0/1176740260 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2415410071 0 0) 0x7f11fc06d1b0 con 0x7f11fc0aa550 2024-05-04T12:19:12.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f11faffd640 1 --2- 172.21.15.52:0/1176740260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11fc15bf50 0x7f11fc15c330 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f11fc0b3ae0 tx=0x7f11ec01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd21c1fd921e70f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:12.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f11f9ffb640 1 -- 172.21.15.52:0/1176740260 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0a7070 con 0x7f11fc15bf50 2024-05-04T12:19:12.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f11f9ffb640 1 -- 172.21.15.52:0/1176740260 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ec0b0070 con 0x7f11fc15bf50 2024-05-04T12:19:12.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f11f9ffb640 1 -- 172.21.15.52:0/1176740260 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0ab890 con 0x7f11fc15bf50 2024-05-04T12:19:12.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 -- 172.21.15.52:0/1176740260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11fc15bf50 msgr2=0x7f11fc15c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 --2- 172.21.15.52:0/1176740260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11fc15bf50 0x7f11fc15c330 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f11fc0b3ae0 tx=0x7f11ec01c450 comp rx=0 tx=0).stop 2024-05-04T12:19:12.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 -- 172.21.15.52:0/1176740260 shutdown_connections 2024-05-04T12:19:12.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 --2- 172.21.15.52:0/1176740260 >> v2:172.21.15.111:3300/0 conn(0x7f11fc0aa0c0 0x7f11fc0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 --2- 172.21.15.52:0/1176740260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11fc15bf50 0x7f11fc15c330 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 -- 172.21.15.52:0/1176740260 >> 172.21.15.52:0/1176740260 conn(0x7f11fc0b14a0 msgr2=0x7f11fc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:12.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 -- 172.21.15.52:0/1176740260 shutdown_connections 2024-05-04T12:19:12.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 -- 172.21.15.52:0/1176740260 wait complete. 2024-05-04T12:19:12.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 Processor -- start 2024-05-04T12:19:12.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 -- start start 2024-05-04T12:19:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11fc0aa0c0 0x7f11fc15afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f11fc0aa550 0x7f11fc14ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11fc15bf50 0x7f11fc147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc169030 con 0x7f11fc15bf50 2024-05-04T12:19:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc0cd4a0 con 0x7f11fc0aa0c0 2024-05-04T12:19:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f12018ba640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f11fc1691b0 con 0x7f11fc0aa550 2024-05-04T12:19:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f11fb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11fc15bf50 0x7f11fc147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f11fb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11fc15bf50 0x7f11fc147610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56684/0 (socket says 172.21.15.52:56684) 2024-05-04T12:19:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.025+0000 7f11fb7fe640 1 -- 172.21.15.52:0/3262329525 learned_addr learned my addr 172.21.15.52:0/3262329525 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11faffd640 1 --2- 172.21.15.52:0/3262329525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11fc0aa0c0 0x7f11fc15afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11fa7fc640 1 --2- 172.21.15.52:0/3262329525 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f11fc0aa550 0x7f11fc14ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11fb7fe640 1 -- 172.21.15.52:0/3262329525 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f11fc0aa550 msgr2=0x7f11fc14ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11fb7fe640 1 --2- 172.21.15.52:0/3262329525 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f11fc0aa550 0x7f11fc14ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11fb7fe640 1 -- 172.21.15.52:0/3262329525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11fc0aa0c0 msgr2=0x7f11fc15afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11fb7fe640 1 --2- 172.21.15.52:0/3262329525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11fc0aa0c0 0x7f11fc15afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11fb7fe640 1 -- 172.21.15.52:0/3262329525 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc15b7d0 con 0x7f11fc15bf50 2024-05-04T12:19:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11fb7fe640 1 --2- 172.21.15.52:0/3262329525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11fc15bf50 0x7f11fc147610 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f11f400c7d0 tx=0x7f11f400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11e3fff640 1 -- 172.21.15.52:0/3262329525 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4018070 con 0x7f11fc15bf50 2024-05-04T12:19:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11e3fff640 1 -- 172.21.15.52:0/3262329525 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f400a640 con 0x7f11fc15bf50 2024-05-04T12:19:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f12018ba640 1 -- 172.21.15.52:0/3262329525 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11fc14f4d0 con 0x7f11fc15bf50 2024-05-04T12:19:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11e3fff640 1 -- 172.21.15.52:0/3262329525 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f400aa70 con 0x7f11fc15bf50 2024-05-04T12:19:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f12018ba640 1 -- 172.21.15.52:0/3262329525 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11fc14fa90 con 0x7f11fc15bf50 2024-05-04T12:19:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f12018ba640 1 -- 172.21.15.52:0/3262329525 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11fc062e50 con 0x7f11fc15bf50 2024-05-04T12:19:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11e3fff640 1 -- 172.21.15.52:0/3262329525 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f11f401b070 con 0x7f11fc15bf50 2024-05-04T12:19:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11e3fff640 1 --2- 172.21.15.52:0/3262329525 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f11e80678c0 0x7f11e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11faffd640 1 --2- 172.21.15.52:0/3262329525 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f11e80678c0 0x7f11e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11e3fff640 1 -- 172.21.15.52:0/3262329525 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7f11f4085370 con 0x7f11fc15bf50 2024-05-04T12:19:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.029+0000 7f11faffd640 1 --2- 172.21.15.52:0/3262329525 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f11e80678c0 0x7f11e8069d80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0af000 tx=0x7f11ec0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:12.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.033+0000 7f11e3fff640 1 -- 172.21.15.52:0/3262329525 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f11f4055e20 con 0x7f11fc15bf50 2024-05-04T12:19:12.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.197+0000 7f12018ba640 1 -- 172.21.15.52:0/3262329525 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11fc15b500 con 0x7f11fc15bf50 2024-05-04T12:19:12.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.201+0000 7f11e3fff640 1 -- 172.21.15.52:0/3262329525 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16913 (secure 0 0 0) 0x7f11fc15b500 con 0x7f11fc15bf50 2024-05-04T12:19:12.204 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:12.204 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:19:04.046290+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:18:20.636173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T12:18:21.180614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:19:12.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.201+0000 7f12018ba640 1 -- 172.21.15.52:0/3262329525 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f11e80678c0 msgr2=0x7f11e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.201+0000 7f12018ba640 1 --2- 172.21.15.52:0/3262329525 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f11e80678c0 0x7f11e8069d80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0af000 tx=0x7f11ec0a7000 comp rx=0 tx=0).stop 2024-05-04T12:19:12.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.201+0000 7f12018ba640 1 -- 172.21.15.52:0/3262329525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11fc15bf50 msgr2=0x7f11fc147610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.201+0000 7f12018ba640 1 --2- 172.21.15.52:0/3262329525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11fc15bf50 0x7f11fc147610 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f11f400c7d0 tx=0x7f11f400cc90 comp rx=0 tx=0).stop 2024-05-04T12:19:12.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.201+0000 7f12018ba640 1 -- 172.21.15.52:0/3262329525 shutdown_connections 2024-05-04T12:19:12.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.201+0000 7f12018ba640 1 --2- 172.21.15.52:0/3262329525 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f11e80678c0 0x7f11e8069d80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.201+0000 7f12018ba640 1 --2- 172.21.15.52:0/3262329525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11fc15bf50 0x7f11fc147610 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.201+0000 7f12018ba640 1 --2- 172.21.15.52:0/3262329525 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f11fc0aa550 0x7f11fc14ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.201+0000 7f12018ba640 1 --2- 172.21.15.52:0/3262329525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11fc0aa0c0 0x7f11fc15afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.201+0000 7f12018ba640 1 -- 172.21.15.52:0/3262329525 >> 172.21.15.52:0/3262329525 conn(0x7f11fc0b14a0 msgr2=0x7f11fc0b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:12.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.201+0000 7f12018ba640 1 -- 172.21.15.52:0/3262329525 shutdown_connections 2024-05-04T12:19:12.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.201+0000 7f12018ba640 1 -- 172.21.15.52:0/3262329525 wait complete. 2024-05-04T12:19:12.223 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-04T12:19:12.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd3da6c640 1 Processor -- start 2024-05-04T12:19:12.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd3da6c640 1 -- start start 2024-05-04T12:19:12.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd3da6c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7efd3815bf50 0x7efd3815c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd3da6c640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd3806d1b0 con 0x7efd380cd260 2024-05-04T12:19:12.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd3da6c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd3806d490 con 0x7efd380aa0c0 2024-05-04T12:19:12.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd3da6c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7efd3806d770 con 0x7efd3815bf50 2024-05-04T12:19:12.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd37fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7efd380cd260 0x7efd3806e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46910/0 (socket says 172.21.15.52:46910) 2024-05-04T12:19:12.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd37fff640 1 -- 172.21.15.52:0/4243153739 learned_addr learned my addr 172.21.15.52:0/4243153739 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:12.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd377fe640 1 --2- 172.21.15.52:0/4243153739 >> v2:172.21.15.111:3300/0 conn(0x7efd3815bf50 0x7efd3815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd367fc640 1 -- 172.21.15.52:0/4243153739 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3726692115 0 0) 0x7efd3806d1b0 con 0x7efd380cd260 2024-05-04T12:19:12.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd367fc640 1 -- 172.21.15.52:0/4243153739 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd24003530 con 0x7efd380cd260 2024-05-04T12:19:12.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd367fc640 1 -- 172.21.15.52:0/4243153739 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3359046755 0 0) 0x7efd24003530 con 0x7efd380cd260 2024-05-04T12:19:12.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd367fc640 1 -- 172.21.15.52:0/4243153739 >> v2:172.21.15.111:3300/0 conn(0x7efd3815bf50 msgr2=0x7efd3815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:12.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd367fc640 1 --2- 172.21.15.52:0/4243153739 >> v2:172.21.15.111:3300/0 conn(0x7efd3815bf50 0x7efd3815c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd367fc640 1 -- 172.21.15.52:0/4243153739 >> v1:172.21.15.111:6789/0 conn(0x7efd380aa0c0 legacy=0x7efd380ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd367fc640 1 -- 172.21.15.52:0/4243153739 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3806da50 con 0x7efd380cd260 2024-05-04T12:19:12.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd367fc640 1 -- 172.21.15.52:0/4243153739 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7efd30001a00 con 0x7efd380cd260 2024-05-04T12:19:12.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd367fc640 1 -- 172.21.15.52:0/4243153739 >> v1:172.21.15.52:6789/0 conn(0x7efd380cd260 legacy=0x7efd3806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd367fc640 1 --2- 172.21.15.52:0/4243153739 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd240039c0 0x7efd24003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd367fc640 1 --2- 172.21.15.52:0/4243153739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd24004810 0x7efd24004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.341+0000 7efd367fc640 1 --2- 172.21.15.52:0/4243153739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd240051d0 0x7efd2400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd367fc640 1 -- 172.21.15.52:0/4243153739 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efd3806d770 con 0x7efd24004810 2024-05-04T12:19:12.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd367fc640 1 -- 172.21.15.52:0/4243153739 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efd2400c070 con 0x7efd240051d0 2024-05-04T12:19:12.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd367fc640 1 -- 172.21.15.52:0/4243153739 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7efd2400c270 con 0x7efd240039c0 2024-05-04T12:19:12.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd36ffd640 1 --2- 172.21.15.52:0/4243153739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd24004810 0x7efd24004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd377fe640 1 --2- 172.21.15.52:0/4243153739 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd240039c0 0x7efd24003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd37fff640 1 --2- 172.21.15.52:0/4243153739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd240051d0 0x7efd2400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd36ffd640 1 -- 172.21.15.52:0/4243153739 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd240039c0 msgr2=0x7efd24003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:12.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd36ffd640 1 --2- 172.21.15.52:0/4243153739 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd240039c0 0x7efd24003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd36ffd640 1 -- 172.21.15.52:0/4243153739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd240051d0 msgr2=0x7efd2400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:12.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd36ffd640 1 --2- 172.21.15.52:0/4243153739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd240051d0 0x7efd2400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd36ffd640 1 -- 172.21.15.52:0/4243153739 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd24003530 con 0x7efd24004810 2024-05-04T12:19:12.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd37fff640 1 -- 172.21.15.52:0/4243153739 reap_dead start 2024-05-04T12:19:12.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd36ffd640 1 --2- 172.21.15.52:0/4243153739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd24004810 0x7efd24004c90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efd28001780 tx=0x7efd2801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6f25204e23ce78 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:12.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd367fc640 1 -- 172.21.15.52:0/4243153739 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd280013a0 con 0x7efd24004810 2024-05-04T12:19:12.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd367fc640 1 -- 172.21.15.52:0/4243153739 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd280052f0 con 0x7efd24004810 2024-05-04T12:19:12.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd367fc640 1 -- 172.21.15.52:0/4243153739 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28003840 con 0x7efd24004810 2024-05-04T12:19:12.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd3da6c640 1 -- 172.21.15.52:0/4243153739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd24004810 msgr2=0x7efd24004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd3da6c640 1 --2- 172.21.15.52:0/4243153739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd24004810 0x7efd24004c90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efd28001780 tx=0x7efd2801c400 comp rx=0 tx=0).stop 2024-05-04T12:19:12.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd3da6c640 1 -- 172.21.15.52:0/4243153739 shutdown_connections 2024-05-04T12:19:12.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd3da6c640 1 --2- 172.21.15.52:0/4243153739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd24004810 0x7efd24004c90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd3da6c640 1 -- 172.21.15.52:0/4243153739 >> 172.21.15.52:0/4243153739 conn(0x7efd380b14a0 msgr2=0x7efd380b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:12.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd3da6c640 1 -- 172.21.15.52:0/4243153739 shutdown_connections 2024-05-04T12:19:12.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.345+0000 7efd3da6c640 1 -- 172.21.15.52:0/4243153739 wait complete. 2024-05-04T12:19:12.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd3da6c640 1 Processor -- start 2024-05-04T12:19:12.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd3da6c640 1 -- start start 2024-05-04T12:19:12.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd3da6c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd24004810 0x7efd381e67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd3da6c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd381f6630 0x7efd3814b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd3da6c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3814b740 0x7efd38157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd3da6c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efd3806cee0 con 0x7efd3814b740 2024-05-04T12:19:12.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd3da6c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efd3806cd60 con 0x7efd24004810 2024-05-04T12:19:12.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd3da6c640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7efd380aa550 con 0x7efd381f6630 2024-05-04T12:19:12.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd37fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3814b740 0x7efd38157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd377fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd24004810 0x7efd381e67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd37fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3814b740 0x7efd38157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56698/0 (socket says 172.21.15.52:56698) 2024-05-04T12:19:12.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd37fff640 1 -- 172.21.15.52:0/1746632127 learned_addr learned my addr 172.21.15.52:0/1746632127 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:12.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd36ffd640 1 --2- 172.21.15.52:0/1746632127 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd381f6630 0x7efd3814b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd37fff640 1 -- 172.21.15.52:0/1746632127 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd381f6630 msgr2=0x7efd3814b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd37fff640 1 --2- 172.21.15.52:0/1746632127 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd381f6630 0x7efd3814b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd37fff640 1 -- 172.21.15.52:0/1746632127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd24004810 msgr2=0x7efd381e67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd37fff640 1 --2- 172.21.15.52:0/1746632127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd24004810 0x7efd381e67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd37fff640 1 -- 172.21.15.52:0/1746632127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3814f300 con 0x7efd3814b740 2024-05-04T12:19:12.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd37fff640 1 --2- 172.21.15.52:0/1746632127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3814b740 0x7efd38157ff0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efd30004c00 tx=0x7efd30010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:12.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd2ffff640 1 -- 172.21.15.52:0/1746632127 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3001a070 con 0x7efd3814b740 2024-05-04T12:19:12.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd3da6c640 1 -- 172.21.15.52:0/1746632127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3814f5e0 con 0x7efd3814b740 2024-05-04T12:19:12.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd2ffff640 1 -- 172.21.15.52:0/1746632127 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd30011040 con 0x7efd3814b740 2024-05-04T12:19:12.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd2ffff640 1 -- 172.21.15.52:0/1746632127 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3000ece0 con 0x7efd3814b740 2024-05-04T12:19:12.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.349+0000 7efd3da6c640 1 -- 172.21.15.52:0/1746632127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3814fba0 con 0x7efd3814b740 2024-05-04T12:19:12.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.353+0000 7efd3da6c640 1 -- 172.21.15.52:0/1746632127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd38062e50 con 0x7efd3814b740 2024-05-04T12:19:12.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.353+0000 7efd2ffff640 1 -- 172.21.15.52:0/1746632127 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efd3001d070 con 0x7efd3814b740 2024-05-04T12:19:12.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.353+0000 7efd2ffff640 1 --2- 172.21.15.52:0/1746632127 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7efd180678f0 0x7efd18069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.353+0000 7efd2ffff640 1 -- 172.21.15.52:0/1746632127 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5854+0+0 (secure 0 0 0) 0x7efd30088530 con 0x7efd3814b740 2024-05-04T12:19:12.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.353+0000 7efd377fe640 1 --2- 172.21.15.52:0/1746632127 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7efd180678f0 0x7efd18069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.353+0000 7efd377fe640 1 --2- 172.21.15.52:0/1746632127 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7efd180678f0 0x7efd18069db0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efd381f5920 tx=0x7efd20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:12.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.357+0000 7efd2ffff640 1 -- 172.21.15.52:0/1746632127 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efd30058870 con 0x7efd3814b740 2024-05-04T12:19:12.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.553+0000 7efd3da6c640 1 -- 172.21.15.52:0/1746632127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efd38075320 con 0x7efd3814b740 2024-05-04T12:19:12.557 INFO:tasks.ceph.mon.a.smithi052.stderr:2024-05-04T12:19:12.553+0000 7fc9c3c17640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-04T12:19:12.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.557+0000 7efd2ffff640 1 -- 172.21.15.52:0/1746632127 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7efd3005cb70 con 0x7efd3814b740 2024-05-04T12:19:12.560 INFO:teuthology.orchestra.run.smithi052.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-04T12:19:12.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.561+0000 7efd3da6c640 1 -- 172.21.15.52:0/1746632127 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7efd180678f0 msgr2=0x7efd18069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.561+0000 7efd3da6c640 1 --2- 172.21.15.52:0/1746632127 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7efd180678f0 0x7efd18069db0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efd381f5920 tx=0x7efd20008040 comp rx=0 tx=0).stop 2024-05-04T12:19:12.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.561+0000 7efd3da6c640 1 -- 172.21.15.52:0/1746632127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3814b740 msgr2=0x7efd38157ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.561+0000 7efd3da6c640 1 --2- 172.21.15.52:0/1746632127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3814b740 0x7efd38157ff0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efd30004c00 tx=0x7efd30010d00 comp rx=0 tx=0).stop 2024-05-04T12:19:12.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.561+0000 7efd3da6c640 1 -- 172.21.15.52:0/1746632127 shutdown_connections 2024-05-04T12:19:12.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.561+0000 7efd3da6c640 1 --2- 172.21.15.52:0/1746632127 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7efd180678f0 0x7efd18069db0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.561+0000 7efd3da6c640 1 --2- 172.21.15.52:0/1746632127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3814b740 0x7efd38157ff0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.561+0000 7efd3da6c640 1 --2- 172.21.15.52:0/1746632127 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd381f6630 0x7efd3814b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.561+0000 7efd3da6c640 1 --2- 172.21.15.52:0/1746632127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd24004810 0x7efd381e67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.561+0000 7efd3da6c640 1 -- 172.21.15.52:0/1746632127 >> 172.21.15.52:0/1746632127 conn(0x7efd380b14a0 msgr2=0x7efd38151c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:12.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.561+0000 7efd3da6c640 1 -- 172.21.15.52:0/1746632127 shutdown_connections 2024-05-04T12:19:12.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.561+0000 7efd3da6c640 1 -- 172.21.15.52:0/1746632127 wait complete. 2024-05-04T12:19:12.574 INFO:tasks.ceph.mds.d.smithi111.stderr: -3260> 2024-05-04T12:18:27.070+0000 7fa9b10dd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-04T12:19:12.583 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-04T12:19:12.637 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2024-05-04T12:19:12.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.705+0000 7fc9daccf640 1 Processor -- start 2024-05-04T12:19:12.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.705+0000 7fc9daccf640 1 -- start start 2024-05-04T12:19:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9daccf640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc9d41457e0 0x7fc9d415c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9daccf640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc9d4145bc0 0x7fc9d406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9daccf640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9d4070690 0x7fc9d40cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9daccf640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc9d406d340 con 0x7fc9d4145bc0 2024-05-04T12:19:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9daccf640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fc9d406d620 con 0x7fc9d41457e0 2024-05-04T12:19:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9daccf640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d406d900 con 0x7fc9d4070690 2024-05-04T12:19:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d3fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc9d4145bc0 0x7fc9d406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d3fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc9d4145bc0 0x7fc9d406caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56700/0 (socket says 172.21.15.52:56700) 2024-05-04T12:19:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d3fff640 1 -- 172.21.15.52:0/2059792633 learned_addr learned my addr 172.21.15.52:0/2059792633 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d926c640 1 --2- 172.21.15.52:0/2059792633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9d4070690 0x7fc9d40cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d8a6b640 1 --2- 172.21.15.52:0/2059792633 >> v2:172.21.15.111:3300/0 conn(0x7fc9d41457e0 0x7fc9d415c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d3fff640 1 -- 172.21.15.52:0/2059792633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9d4070690 msgr2=0x7fc9d40cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d3fff640 1 --2- 172.21.15.52:0/2059792633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9d4070690 0x7fc9d40cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d3fff640 1 -- 172.21.15.52:0/2059792633 >> v2:172.21.15.111:3300/0 conn(0x7fc9d41457e0 msgr2=0x7fc9d415c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d3fff640 1 --2- 172.21.15.52:0/2059792633 >> v2:172.21.15.111:3300/0 conn(0x7fc9d41457e0 0x7fc9d415c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d3fff640 1 -- 172.21.15.52:0/2059792633 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d406dbe0 con 0x7fc9d4145bc0 2024-05-04T12:19:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d8a6b640 1 --2- 172.21.15.52:0/2059792633 >> v2:172.21.15.111:3300/0 conn(0x7fc9d41457e0 0x7fc9d415c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:12.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d3fff640 1 --2- 172.21.15.52:0/2059792633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9d4145bc0 0x7fc9d406caf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc9d406b280 tx=0x7fc9c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b892483f35442b5f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:12.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d37fe640 1 -- 172.21.15.52:0/2059792633 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80b0070 con 0x7fc9d4145bc0 2024-05-04T12:19:12.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d37fe640 1 -- 172.21.15.52:0/2059792633 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c80ab600 con 0x7fc9d4145bc0 2024-05-04T12:19:12.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.709+0000 7fc9d37fe640 1 -- 172.21.15.52:0/2059792633 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80ab9d0 con 0x7fc9d4145bc0 2024-05-04T12:19:12.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 -- 172.21.15.52:0/2059792633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9d4145bc0 msgr2=0x7fc9d406caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 --2- 172.21.15.52:0/2059792633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9d4145bc0 0x7fc9d406caf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc9d406b280 tx=0x7fc9c801c520 comp rx=0 tx=0).stop 2024-05-04T12:19:12.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 -- 172.21.15.52:0/2059792633 shutdown_connections 2024-05-04T12:19:12.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 --2- 172.21.15.52:0/2059792633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9d4070690 0x7fc9d40cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 --2- 172.21.15.52:0/2059792633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9d4145bc0 0x7fc9d406caf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 --2- 172.21.15.52:0/2059792633 >> v2:172.21.15.111:3300/0 conn(0x7fc9d41457e0 0x7fc9d415c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 -- 172.21.15.52:0/2059792633 >> 172.21.15.52:0/2059792633 conn(0x7fc9d40b1300 msgr2=0x7fc9d40b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:12.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 -- 172.21.15.52:0/2059792633 shutdown_connections 2024-05-04T12:19:12.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 -- 172.21.15.52:0/2059792633 wait complete. 2024-05-04T12:19:12.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 Processor -- start 2024-05-04T12:19:12.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 -- start start 2024-05-04T12:19:12.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9d4070690 0x7fc9d415a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9d41457e0 0x7fc9d40ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9d4145bc0 0x7fc9d40ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d406ef80 con 0x7fc9d4145bc0 2024-05-04T12:19:12.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d406ee00 con 0x7fc9d41457e0 2024-05-04T12:19:12.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d406f100 con 0x7fc9d4070690 2024-05-04T12:19:12.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d926c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9d4145bc0 0x7fc9d40ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d926c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9d4145bc0 0x7fc9d40ade10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56708/0 (socket says 172.21.15.52:56708) 2024-05-04T12:19:12.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d926c640 1 -- 172.21.15.52:0/1672548317 learned_addr learned my addr 172.21.15.52:0/1672548317 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:12.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d8a6b640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9d4070690 0x7fc9d415a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d3fff640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9d41457e0 0x7fc9d40ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d926c640 1 -- 172.21.15.52:0/1672548317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9d4070690 msgr2=0x7fc9d415a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d926c640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9d4070690 0x7fc9d415a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d926c640 1 -- 172.21.15.52:0/1672548317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9d41457e0 msgr2=0x7fc9d40ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:12.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d926c640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9d41457e0 0x7fc9d40ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:12.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d926c640 1 -- 172.21.15.52:0/1672548317 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d41411e0 con 0x7fc9d4145bc0 2024-05-04T12:19:12.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d3fff640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9d41457e0 0x7fc9d40ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:12.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d926c640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9d4145bc0 0x7fc9d40ade10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc007e60 tx=0x7fc9cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:12.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d17fa640 1 -- 172.21.15.52:0/1672548317 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc014420 con 0x7fc9d4145bc0 2024-05-04T12:19:12.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d17fa640 1 -- 172.21.15.52:0/1672548317 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9cc019070 con 0x7fc9d4145bc0 2024-05-04T12:19:12.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9d17fa640 1 -- 172.21.15.52:0/1672548317 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc014820 con 0x7fc9d4145bc0 2024-05-04T12:19:12.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 -- 172.21.15.52:0/1672548317 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d40ccfa0 con 0x7fc9d4145bc0 2024-05-04T12:19:12.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 -- 172.21.15.52:0/1672548317 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d40cd560 con 0x7fc9d4145bc0 2024-05-04T12:19:12.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.713+0000 7fc9daccf640 1 -- 172.21.15.52:0/1672548317 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d41413b0 con 0x7fc9d4145bc0 2024-05-04T12:19:12.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.717+0000 7fc9d17fa640 1 -- 172.21.15.52:0/1672548317 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc9cc0149c0 con 0x7fc9d4145bc0 2024-05-04T12:19:12.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.717+0000 7fc9d17fa640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc9c00678f0 0x7fc9c0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:12.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.717+0000 7fc9d17fa640 1 -- 172.21.15.52:0/1672548317 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5940+0+0 (secure 0 0 0) 0x7fc9cc086590 con 0x7fc9d4145bc0 2024-05-04T12:19:12.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.717+0000 7fc9d8a6b640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc9c00678f0 0x7fc9c0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:12.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.717+0000 7fc9d8a6b640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc9c00678f0 0x7fc9c0069db0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400d8d0 tx=0x7fc9c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:12.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.721+0000 7fc9d17fa640 1 -- 172.21.15.52:0/1672548317 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc9cc056fe0 con 0x7fc9d4145bc0 2024-05-04T12:19:12.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:12.917+0000 7fc9daccf640 1 -- 172.21.15.52:0/1672548317 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc9d4075320 con 0x7fc9d4145bc0 2024-05-04T12:19:13.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.565+0000 7fc9d17fa640 1 -- 172.21.15.52:0/1672548317 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7fc9cc05be20 con 0x7fc9d4145bc0 2024-05-04T12:19:13.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.565+0000 7fc9daccf640 1 -- 172.21.15.52:0/1672548317 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc9c00678f0 msgr2=0x7fc9c0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:13.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.565+0000 7fc9daccf640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc9c00678f0 0x7fc9c0069db0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400d8d0 tx=0x7fc9c4008040 comp rx=0 tx=0).stop 2024-05-04T12:19:13.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.565+0000 7fc9daccf640 1 -- 172.21.15.52:0/1672548317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9d4145bc0 msgr2=0x7fc9d40ade10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:13.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.565+0000 7fc9daccf640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9d4145bc0 0x7fc9d40ade10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc007e60 tx=0x7fc9cc00e4b0 comp rx=0 tx=0).stop 2024-05-04T12:19:13.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.569+0000 7fc9daccf640 1 -- 172.21.15.52:0/1672548317 shutdown_connections 2024-05-04T12:19:13.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.569+0000 7fc9daccf640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc9c00678f0 0x7fc9c0069db0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:13.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.569+0000 7fc9daccf640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9d4145bc0 0x7fc9d40ade10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:13.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.569+0000 7fc9daccf640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9d41457e0 0x7fc9d40ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:13.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.569+0000 7fc9daccf640 1 --2- 172.21.15.52:0/1672548317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc9d4070690 0x7fc9d415a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:13.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.569+0000 7fc9daccf640 1 -- 172.21.15.52:0/1672548317 >> 172.21.15.52:0/1672548317 conn(0x7fc9d40b1300 msgr2=0x7fc9d415ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:13.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.569+0000 7fc9daccf640 1 -- 172.21.15.52:0/1672548317 shutdown_connections 2024-05-04T12:19:13.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.569+0000 7fc9daccf640 1 -- 172.21.15.52:0/1672548317 wait complete. 2024-05-04T12:19:13.591 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-04T12:19:13.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.717+0000 7fb3066e2640 1 Processor -- start 2024-05-04T12:19:13.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.717+0000 7fb3066e2640 1 -- start start 2024-05-04T12:19:13.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb3066e2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3001608f0 0x7fb30006ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:13.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb3066e2640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb30006f390 0x7fb3000ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:13.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb3066e2640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3000ced50 con 0x7fb300144fc0 2024-05-04T12:19:13.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb3066e2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fb3001453a0 con 0x7fb3001608f0 2024-05-04T12:19:13.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb3066e2640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fb30015fbc0 con 0x7fb30006f390 2024-05-04T12:19:13.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb2ff7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3001608f0 0x7fb30006ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:13.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb304c7f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb30006f390 0x7fb3000ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:13.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb2ff7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3001608f0 0x7fb30006ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:34730/0 (socket says 172.21.15.52:34730) 2024-05-04T12:19:13.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb2ff7fe640 1 -- 172.21.15.52:0/2880188645 learned_addr learned my addr 172.21.15.52:0/2880188645 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:13.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb2fffff640 1 --1- 172.21.15.52:0/2880188645 >> v1:172.21.15.111:6789/0 conn(0x7fb300144fc0 0x7fb3000ce300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.52:34866/0 (socket says 172.21.15.52:34866) 2024-05-04T12:19:13.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb304c7f640 1 -- 172.21.15.52:0/2880188645 >> v2:172.21.15.111:3300/0 conn(0x7fb3001608f0 msgr2=0x7fb30006ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:13.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb304c7f640 1 --2- 172.21.15.52:0/2880188645 >> v2:172.21.15.111:3300/0 conn(0x7fb3001608f0 0x7fb30006ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:13.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb304c7f640 1 -- 172.21.15.52:0/2880188645 >> v1:172.21.15.111:6789/0 conn(0x7fb300144fc0 legacy=0x7fb3000ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:13.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb304c7f640 1 -- 172.21.15.52:0/2880188645 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30015fe10 con 0x7fb30006f390 2024-05-04T12:19:13.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb304c7f640 1 --2- 172.21.15.52:0/2880188645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb30006f390 0x7fb3000ccac0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009d90 tx=0x7fb2f801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baade06cf01ceff server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:13.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb2feffd640 1 -- 172.21.15.52:0/2880188645 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80b2070 con 0x7fb30006f390 2024-05-04T12:19:13.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb2feffd640 1 -- 172.21.15.52:0/2880188645 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2f80a5890 con 0x7fb30006f390 2024-05-04T12:19:13.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.721+0000 7fb2feffd640 1 -- 172.21.15.52:0/2880188645 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80a5c60 con 0x7fb30006f390 2024-05-04T12:19:13.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 -- 172.21.15.52:0/2880188645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb30006f390 msgr2=0x7fb3000ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:13.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 --2- 172.21.15.52:0/2880188645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb30006f390 0x7fb3000ccac0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009d90 tx=0x7fb2f801c8e0 comp rx=0 tx=0).stop 2024-05-04T12:19:13.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 -- 172.21.15.52:0/2880188645 shutdown_connections 2024-05-04T12:19:13.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 --2- 172.21.15.52:0/2880188645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb30006f390 0x7fb3000ccac0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:13.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 --2- 172.21.15.52:0/2880188645 >> v2:172.21.15.111:3300/0 conn(0x7fb3001608f0 0x7fb30006ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:13.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 -- 172.21.15.52:0/2880188645 >> 172.21.15.52:0/2880188645 conn(0x7fb300067760 msgr2=0x7fb3000b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:13.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 -- 172.21.15.52:0/2880188645 shutdown_connections 2024-05-04T12:19:13.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 -- 172.21.15.52:0/2880188645 wait complete. 2024-05-04T12:19:13.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 Processor -- start 2024-05-04T12:19:13.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 -- start start 2024-05-04T12:19:13.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb30006f390 0x7fb30013ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:13.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb300144fc0 0x7fb300142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:13.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3001608f0 0x7fb300159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:13.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb300072250 con 0x7fb30006f390 2024-05-04T12:19:13.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb3000720d0 con 0x7fb3001608f0 2024-05-04T12:19:13.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fb300168f60 con 0x7fb300144fc0 2024-05-04T12:19:13.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb2fffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb30006f390 0x7fb30013ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:13.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb2fffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb30006f390 0x7fb30013ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56716/0 (socket says 172.21.15.52:56716) 2024-05-04T12:19:13.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb2fffff640 1 -- 172.21.15.52:0/4065569469 learned_addr learned my addr 172.21.15.52:0/4065569469 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:13.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb304c7f640 1 --2- 172.21.15.52:0/4065569469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3001608f0 0x7fb300159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:13.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb2ff7fe640 1 --2- 172.21.15.52:0/4065569469 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb300144fc0 0x7fb300142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:13.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb2fffff640 1 -- 172.21.15.52:0/4065569469 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb300144fc0 msgr2=0x7fb300142940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:13.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb2fffff640 1 --2- 172.21.15.52:0/4065569469 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb300144fc0 0x7fb300142940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:13.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb2fffff640 1 -- 172.21.15.52:0/4065569469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3001608f0 msgr2=0x7fb300159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:13.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb2fffff640 1 --2- 172.21.15.52:0/4065569469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3001608f0 0x7fb300159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:13.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb2fffff640 1 -- 172.21.15.52:0/4065569469 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb300142e80 con 0x7fb30006f390 2024-05-04T12:19:13.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb2fffff640 1 --2- 172.21.15.52:0/4065569469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb30006f390 0x7fb30013ff10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800c420 tx=0x7fb2e800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:13.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb2fcff9640 1 -- 172.21.15.52:0/4065569469 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80074e0 con 0x7fb30006f390 2024-05-04T12:19:13.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb2fcff9640 1 -- 172.21.15.52:0/4065569469 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e8007680 con 0x7fb30006f390 2024-05-04T12:19:13.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 -- 172.21.15.52:0/4065569469 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb300143150 con 0x7fb30006f390 2024-05-04T12:19:13.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb2fcff9640 1 -- 172.21.15.52:0/4065569469 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8004a30 con 0x7fb30006f390 2024-05-04T12:19:13.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.725+0000 7fb3066e2640 1 -- 172.21.15.52:0/4065569469 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb300143710 con 0x7fb30006f390 2024-05-04T12:19:13.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.729+0000 7fb3066e2640 1 -- 172.21.15.52:0/4065569469 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb300062e50 con 0x7fb30006f390 2024-05-04T12:19:13.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.729+0000 7fb2fcff9640 1 -- 172.21.15.52:0/4065569469 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb2e801a020 con 0x7fb30006f390 2024-05-04T12:19:13.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.729+0000 7fb2fcff9640 1 --2- 172.21.15.52:0/4065569469 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb2e00678c0 0x7fb2e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:13.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.729+0000 7fb2fcff9640 1 -- 172.21.15.52:0/4065569469 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5940+0+0 (secure 0 0 0) 0x7fb2e80853d0 con 0x7fb30006f390 2024-05-04T12:19:13.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.729+0000 7fb2ff7fe640 1 --2- 172.21.15.52:0/4065569469 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb2e00678c0 0x7fb2e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:13.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.729+0000 7fb2ff7fe640 1 --2- 172.21.15.52:0/4065569469 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb2e00678c0 0x7fb2e0069d80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009b50 tx=0x7fb2f0009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:13.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.733+0000 7fb2fcff9640 1 -- 172.21.15.52:0/4065569469 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb2e8055860 con 0x7fb30006f390 2024-05-04T12:19:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:13.897+0000 7fb3066e2640 1 -- 172.21.15.52:0/4065569469 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb3000719b0 con 0x7fb30006f390 2024-05-04T12:19:14.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.565+0000 7fb2fcff9640 1 -- 172.21.15.52:0/4065569469 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7fb2e8059b60 con 0x7fb30006f390 2024-05-04T12:19:14.571 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' removed 2024-05-04T12:19:14.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.569+0000 7fb3066e2640 1 -- 172.21.15.52:0/4065569469 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb2e00678c0 msgr2=0x7fb2e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:14.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.569+0000 7fb3066e2640 1 --2- 172.21.15.52:0/4065569469 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb2e00678c0 0x7fb2e0069d80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009b50 tx=0x7fb2f0009240 comp rx=0 tx=0).stop 2024-05-04T12:19:14.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.569+0000 7fb3066e2640 1 -- 172.21.15.52:0/4065569469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb30006f390 msgr2=0x7fb30013ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:14.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.569+0000 7fb3066e2640 1 --2- 172.21.15.52:0/4065569469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb30006f390 0x7fb30013ff10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800c420 tx=0x7fb2e800c8e0 comp rx=0 tx=0).stop 2024-05-04T12:19:14.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.569+0000 7fb3066e2640 1 -- 172.21.15.52:0/4065569469 shutdown_connections 2024-05-04T12:19:14.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.569+0000 7fb3066e2640 1 --2- 172.21.15.52:0/4065569469 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb2e00678c0 0x7fb2e0069d80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:14.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.569+0000 7fb3066e2640 1 --2- 172.21.15.52:0/4065569469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3001608f0 0x7fb300159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:14.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.569+0000 7fb3066e2640 1 --2- 172.21.15.52:0/4065569469 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb300144fc0 0x7fb300142940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:14.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.569+0000 7fb3066e2640 1 --2- 172.21.15.52:0/4065569469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb30006f390 0x7fb30013ff10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:14.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.569+0000 7fb3066e2640 1 -- 172.21.15.52:0/4065569469 >> 172.21.15.52:0/4065569469 conn(0x7fb300067760 msgr2=0x7fb30013e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.569+0000 7fb3066e2640 1 -- 172.21.15.52:0/4065569469 shutdown_connections 2024-05-04T12:19:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.569+0000 7fb3066e2640 1 -- 172.21.15.52:0/4065569469 wait complete. 2024-05-04T12:19:14.606 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-04T12:19:14.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.729+0000 7fe71fcf3640 1 Processor -- start 2024-05-04T12:19:14.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.729+0000 7fe71fcf3640 1 -- start start 2024-05-04T12:19:14.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.729+0000 7fe71fcf3640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe718144fc0 0x7fe7180ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:14.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.729+0000 7fe71fcf3640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe71806f560 0x7fe7180ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:14.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.729+0000 7fe71fcf3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7180ced50 con 0x7fe7181608f0 2024-05-04T12:19:14.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.729+0000 7fe71fcf3640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fe7181453a0 con 0x7fe71806f560 2024-05-04T12:19:14.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.729+0000 7fe71fcf3640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe71815fbc0 con 0x7fe718144fc0 2024-05-04T12:19:14.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71e290640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe71806f560 0x7fe7180ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:14.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71e290640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe71806f560 0x7fe7180ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56718/0 (socket says 172.21.15.52:56718) 2024-05-04T12:19:14.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71e290640 1 -- 172.21.15.52:0/1007063635 learned_addr learned my addr 172.21.15.52:0/1007063635 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:14.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71da8f640 1 --2- 172.21.15.52:0/1007063635 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe718144fc0 0x7fe7180ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:14.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71e290640 1 -- 172.21.15.52:0/1007063635 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe718144fc0 msgr2=0x7fe7180ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:14.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71e290640 1 --2- 172.21.15.52:0/1007063635 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe718144fc0 0x7fe7180ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:14.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71e290640 1 -- 172.21.15.52:0/1007063635 >> v1:172.21.15.111:6789/0 conn(0x7fe7181608f0 legacy=0x7fe71806ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:14.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71e290640 1 -- 172.21.15.52:0/1007063635 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71815fe10 con 0x7fe71806f560 2024-05-04T12:19:14.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71da8f640 1 --2- 172.21.15.52:0/1007063635 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe718144fc0 0x7fe7180ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:14.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71e290640 1 --2- 172.21.15.52:0/1007063635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe71806f560 0x7fe7180ccac0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe710009870 tx=0x7fe71001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22eeffaa661e595 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:14.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71ca8d640 1 -- 172.21.15.52:0/1007063635 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7100aa600 con 0x7fe71806f560 2024-05-04T12:19:14.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71ca8d640 1 -- 172.21.15.52:0/1007063635 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7100af070 con 0x7fe71806f560 2024-05-04T12:19:14.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71ca8d640 1 -- 172.21.15.52:0/1007063635 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7100aa9c0 con 0x7fe71806f560 2024-05-04T12:19:14.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71fcf3640 1 -- 172.21.15.52:0/1007063635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe71806f560 msgr2=0x7fe7180ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:14.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71fcf3640 1 --2- 172.21.15.52:0/1007063635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe71806f560 0x7fe7180ccac0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe710009870 tx=0x7fe71001c4a0 comp rx=0 tx=0).stop 2024-05-04T12:19:14.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71fcf3640 1 -- 172.21.15.52:0/1007063635 shutdown_connections 2024-05-04T12:19:14.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71fcf3640 1 --2- 172.21.15.52:0/1007063635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe71806f560 0x7fe7180ccac0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:14.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71fcf3640 1 --2- 172.21.15.52:0/1007063635 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe718144fc0 0x7fe7180ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:14.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71fcf3640 1 -- 172.21.15.52:0/1007063635 >> 172.21.15.52:0/1007063635 conn(0x7fe718067760 msgr2=0x7fe7180b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:14.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71fcf3640 1 -- 172.21.15.52:0/1007063635 shutdown_connections 2024-05-04T12:19:14.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.733+0000 7fe71fcf3640 1 -- 172.21.15.52:0/1007063635 wait complete. 2024-05-04T12:19:14.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71fcf3640 1 Processor -- start 2024-05-04T12:19:14.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71fcf3640 1 -- start start 2024-05-04T12:19:14.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71fcf3640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe71806f560 0x7fe718137450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:14.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71fcf3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe718144fc0 0x7fe71813c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:14.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71fcf3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7181608f0 0x7fe71815c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:14.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71fcf3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe7181690e0 con 0x7fe718144fc0 2024-05-04T12:19:14.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71fcf3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe718168f60 con 0x7fe7181608f0 2024-05-04T12:19:14.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71fcf3640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe718169260 con 0x7fe71806f560 2024-05-04T12:19:14.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71d28e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe718144fc0 0x7fe71813c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:14.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71d28e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe718144fc0 0x7fe71813c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56722/0 (socket says 172.21.15.52:56722) 2024-05-04T12:19:14.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71d28e640 1 -- 172.21.15.52:0/3997421428 learned_addr learned my addr 172.21.15.52:0/3997421428 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:14.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71da8f640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe71806f560 0x7fe718137450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:14.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71e290640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7181608f0 0x7fe71815c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:14.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71d28e640 1 -- 172.21.15.52:0/3997421428 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe71806f560 msgr2=0x7fe718137450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:14.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71d28e640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe71806f560 0x7fe718137450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:14.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71d28e640 1 -- 172.21.15.52:0/3997421428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7181608f0 msgr2=0x7fe71815c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:14.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71d28e640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7181608f0 0x7fe71815c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:14.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71d28e640 1 -- 172.21.15.52:0/3997421428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71813d070 con 0x7fe718144fc0 2024-05-04T12:19:14.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71da8f640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe71806f560 0x7fe718137450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:14.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71d28e640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe718144fc0 0x7fe71813c5f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe70800c3a0 tx=0x7fe70800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:14.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe70e7fc640 1 -- 172.21.15.52:0/3997421428 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7080052d0 con 0x7fe718144fc0 2024-05-04T12:19:14.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe70e7fc640 1 -- 172.21.15.52:0/3997421428 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe708005470 con 0x7fe718144fc0 2024-05-04T12:19:14.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71fcf3640 1 -- 172.21.15.52:0/3997421428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71806cd50 con 0x7fe718144fc0 2024-05-04T12:19:14.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe70e7fc640 1 -- 172.21.15.52:0/3997421428 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708007260 con 0x7fe718144fc0 2024-05-04T12:19:14.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71fcf3640 1 -- 172.21.15.52:0/3997421428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71806d310 con 0x7fe718144fc0 2024-05-04T12:19:14.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.737+0000 7fe71fcf3640 1 -- 172.21.15.52:0/3997421428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe718062e50 con 0x7fe718144fc0 2024-05-04T12:19:14.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.741+0000 7fe70e7fc640 1 -- 172.21.15.52:0/3997421428 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe708007400 con 0x7fe718144fc0 2024-05-04T12:19:14.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.741+0000 7fe70e7fc640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe6fc0678c0 0x7fe6fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:14.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.741+0000 7fe70e7fc640 1 -- 172.21.15.52:0/3997421428 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5502+0+0 (secure 0 0 0) 0x7fe708085100 con 0x7fe718144fc0 2024-05-04T12:19:14.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.741+0000 7fe71da8f640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe6fc0678c0 0x7fe6fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:14.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.741+0000 7fe71da8f640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe6fc0678c0 0x7fe6fc069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe71806f530 tx=0x7fe7140076e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:14.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.745+0000 7fe70e7fc640 1 -- 172.21.15.52:0/3997421428 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe708055c70 con 0x7fe718144fc0 2024-05-04T12:19:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:14.901+0000 7fe71fcf3640 1 -- 172.21.15.52:0/3997421428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe718075320 con 0x7fe718144fc0 2024-05-04T12:19:15.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.573+0000 7fe70e7fc640 1 -- 172.21.15.52:0/3997421428 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7fe708055570 con 0x7fe718144fc0 2024-05-04T12:19:15.579 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' removed 2024-05-04T12:19:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.581+0000 7fe71fcf3640 1 -- 172.21.15.52:0/3997421428 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe6fc0678c0 msgr2=0x7fe6fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.581+0000 7fe71fcf3640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe6fc0678c0 0x7fe6fc069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe71806f530 tx=0x7fe7140076e0 comp rx=0 tx=0).stop 2024-05-04T12:19:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.581+0000 7fe71fcf3640 1 -- 172.21.15.52:0/3997421428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe718144fc0 msgr2=0x7fe71813c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.581+0000 7fe71fcf3640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe718144fc0 0x7fe71813c5f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe70800c3a0 tx=0x7fe70800c860 comp rx=0 tx=0).stop 2024-05-04T12:19:15.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.581+0000 7fe71fcf3640 1 -- 172.21.15.52:0/3997421428 shutdown_connections 2024-05-04T12:19:15.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.581+0000 7fe71fcf3640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe6fc0678c0 0x7fe6fc069d80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:15.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.581+0000 7fe71fcf3640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7181608f0 0x7fe71815c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:15.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.581+0000 7fe71fcf3640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe718144fc0 0x7fe71813c5f0 secure :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe70800c3a0 tx=0x7fe70800c860 comp rx=0 tx=0).stop 2024-05-04T12:19:15.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.581+0000 7fe71fcf3640 1 --2- 172.21.15.52:0/3997421428 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe71806f560 0x7fe718137450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:15.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.581+0000 7fe71fcf3640 1 -- 172.21.15.52:0/3997421428 >> 172.21.15.52:0/3997421428 conn(0x7fe718067760 msgr2=0x7fe7180b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:15.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.581+0000 7fe71fcf3640 1 -- 172.21.15.52:0/3997421428 shutdown_connections 2024-05-04T12:19:15.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.581+0000 7fe71fcf3640 1 -- 172.21.15.52:0/3997421428 wait complete. 2024-05-04T12:19:15.611 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-04T12:19:15.611 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-04T12:19:15.611 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-04T12:19:15.611 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T12:19:15.612 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-04T12:19:15.612 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-04T12:19:15.612 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-04T12:19:15.612 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T12:19:15.612 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-04T12:19:15.612 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-04T12:19:15.612 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-04T12:19:15.612 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T12:19:15.613 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-04T12:19:15.613 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-04T12:19:15.613 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-04T12:19:15.613 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T12:19:15.622 INFO:tasks.ceph.mds.a:Stopped 2024-05-04T12:19:15.622 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-04T12:19:15.624 INFO:tasks.ceph.mds.c:Stopped 2024-05-04T12:19:15.624 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-04T12:19:15.629 INFO:tasks.ceph.mds.d:Stopped 2024-05-04T12:19:15.629 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-04T12:19:15.632 INFO:tasks.ceph.mds.b:Stopped 2024-05-04T12:19:15.632 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-04T12:19:15.672 INFO:tasks.ceph.mds.a:Started 2024-05-04T12:19:15.672 DEBUG:teuthology.parallel:result is None 2024-05-04T12:19:15.673 INFO:tasks.ceph.mds.c:Started 2024-05-04T12:19:15.674 DEBUG:teuthology.parallel:result is None 2024-05-04T12:19:15.674 INFO:tasks.ceph.mds.d:Started 2024-05-04T12:19:15.675 DEBUG:teuthology.parallel:result is None 2024-05-04T12:19:15.676 INFO:tasks.ceph.mds.b:Started 2024-05-04T12:19:15.676 DEBUG:teuthology.parallel:result is None 2024-05-04T12:19:15.677 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-04T12:19:15.734 INFO:tasks.ceph.mds.a.smithi052.stdout:starting mds.a at 2024-05-04T12:19:15.743 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2024-05-04T12:19:15.744 INFO:tasks.ceph.mds.c.smithi052.stdout:starting mds.c at 2024-05-04T12:19:15.749 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2024-05-04T12:19:15.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65d525b640 1 Processor -- start 2024-05-04T12:19:15.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65d525b640 1 -- start start 2024-05-04T12:19:15.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65d525b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f65d00ceb80 0x7f65d00cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:15.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65d525b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f65d0067070 0x7f65d00674e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:15.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65d525b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65d015d490 con 0x7f65d01424f0 2024-05-04T12:19:15.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65d525b640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f65d0067a20 con 0x7f65d00ceb80 2024-05-04T12:19:15.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65d525b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f65d00cce10 con 0x7f65d0067070 2024-05-04T12:19:15.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65ce59c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f65d00ceb80 0x7f65d00cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:15.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65ce59c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f65d00ceb80 0x7f65d00cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56770/0 (socket says 172.21.15.52:56770) 2024-05-04T12:19:15.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65ce59c640 1 -- 172.21.15.52:0/3610804975 learned_addr learned my addr 172.21.15.52:0/3610804975 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:15.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65cf59e640 1 --2- 172.21.15.52:0/3610804975 >> v2:172.21.15.111:3300/0 conn(0x7f65d0067070 0x7f65d00674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:15.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65ce59c640 1 -- 172.21.15.52:0/3610804975 >> v2:172.21.15.111:3300/0 conn(0x7f65d0067070 msgr2=0x7f65d00674e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:15.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65ce59c640 1 --2- 172.21.15.52:0/3610804975 >> v2:172.21.15.111:3300/0 conn(0x7f65d0067070 0x7f65d00674e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:15.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65ce59c640 1 -- 172.21.15.52:0/3610804975 >> v1:172.21.15.111:6789/0 conn(0x7f65d01424f0 legacy=0x7f65d00cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:15.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65ce59c640 1 -- 172.21.15.52:0/3610804975 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d00cd0f0 con 0x7f65d00ceb80 2024-05-04T12:19:15.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65cf59e640 1 --2- 172.21.15.52:0/3610804975 >> v2:172.21.15.111:3300/0 conn(0x7f65d0067070 0x7f65d00674e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:15.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65ce59c640 1 --2- 172.21.15.52:0/3610804975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65d00ceb80 0x7f65d00cf030 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f65d00b2950 tx=0x7f65c401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5934da13d508a91 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:15.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65cdd9b640 1 -- 172.21.15.52:0/3610804975 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c40b0070 con 0x7f65d00ceb80 2024-05-04T12:19:15.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65cdd9b640 1 -- 172.21.15.52:0/3610804975 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c40ab600 con 0x7f65d00ceb80 2024-05-04T12:19:15.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.801+0000 7f65cdd9b640 1 -- 172.21.15.52:0/3610804975 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c40ab9d0 con 0x7f65d00ceb80 2024-05-04T12:19:15.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 -- 172.21.15.52:0/3610804975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65d00ceb80 msgr2=0x7f65d00cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:15.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 --2- 172.21.15.52:0/3610804975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65d00ceb80 0x7f65d00cf030 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f65d00b2950 tx=0x7f65c401c4e0 comp rx=0 tx=0).stop 2024-05-04T12:19:15.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 -- 172.21.15.52:0/3610804975 shutdown_connections 2024-05-04T12:19:15.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 --2- 172.21.15.52:0/3610804975 >> v2:172.21.15.111:3300/0 conn(0x7f65d0067070 0x7f65d00674e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:15.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 --2- 172.21.15.52:0/3610804975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65d00ceb80 0x7f65d00cf030 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:15.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 -- 172.21.15.52:0/3610804975 >> 172.21.15.52:0/3610804975 conn(0x7f65d00b1c10 msgr2=0x7f65d00b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:15.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 -- 172.21.15.52:0/3610804975 shutdown_connections 2024-05-04T12:19:15.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 -- 172.21.15.52:0/3610804975 wait complete. 2024-05-04T12:19:15.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 Processor -- start 2024-05-04T12:19:15.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 -- start start 2024-05-04T12:19:15.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f65d0067070 0x7f65d0156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:15.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65d00ceb80 0x7f65d013e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:15.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65d01424f0 0x7f65d0153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:15.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f65d015d0b0 con 0x7f65d01424f0 2024-05-04T12:19:15.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f65d015cf30 con 0x7f65d00ceb80 2024-05-04T12:19:15.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f65d015d230 con 0x7f65d0067070 2024-05-04T12:19:15.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65cf59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65d01424f0 0x7f65d0153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:15.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65cf59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65d01424f0 0x7f65d0153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56780/0 (socket says 172.21.15.52:56780) 2024-05-04T12:19:15.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65cf59e640 1 -- 172.21.15.52:0/2066417596 learned_addr learned my addr 172.21.15.52:0/2066417596 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:15.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65ced9d640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f65d0067070 0x7f65d0156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:15.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65ce59c640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65d00ceb80 0x7f65d013e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:15.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65cf59e640 1 -- 172.21.15.52:0/2066417596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f65d0067070 msgr2=0x7f65d0156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:15.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65cf59e640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f65d0067070 0x7f65d0156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:15.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65cf59e640 1 -- 172.21.15.52:0/2066417596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65d00ceb80 msgr2=0x7f65d013e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:15.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65cf59e640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65d00ceb80 0x7f65d013e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:15.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65cf59e640 1 -- 172.21.15.52:0/2066417596 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d015bc50 con 0x7f65d01424f0 2024-05-04T12:19:15.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65ced9d640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f65d0067070 0x7f65d0156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:15.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65cf59e640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65d01424f0 0x7f65d0153e70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f65c800cef0 tx=0x7f65c800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:15.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65b77fe640 1 -- 172.21.15.52:0/2066417596 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c801a070 con 0x7f65d01424f0 2024-05-04T12:19:15.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65b77fe640 1 -- 172.21.15.52:0/2066417596 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c800d860 con 0x7f65d01424f0 2024-05-04T12:19:15.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 -- 172.21.15.52:0/2066417596 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d01564d0 con 0x7f65d01424f0 2024-05-04T12:19:15.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65b77fe640 1 -- 172.21.15.52:0/2066417596 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c800dc50 con 0x7f65d01424f0 2024-05-04T12:19:15.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.805+0000 7f65d525b640 1 -- 172.21.15.52:0/2066417596 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d015bf30 con 0x7f65d01424f0 2024-05-04T12:19:15.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.809+0000 7f65d525b640 1 -- 172.21.15.52:0/2066417596 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d0062e50 con 0x7f65d01424f0 2024-05-04T12:19:15.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.809+0000 7f65b77fe640 1 -- 172.21.15.52:0/2066417596 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f65c801d070 con 0x7f65d01424f0 2024-05-04T12:19:15.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.809+0000 7f65b77fe640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f65bc0678c0 0x7f65bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:15.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.809+0000 7f65b77fe640 1 -- 172.21.15.52:0/2066417596 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4943+0+0 (secure 0 0 0) 0x7f65c8086110 con 0x7f65d01424f0 2024-05-04T12:19:15.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.809+0000 7f65ced9d640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f65bc0678c0 0x7f65bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:15.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.809+0000 7f65ced9d640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f65bc0678c0 0x7f65bc069d80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f65d01550d0 tx=0x7f65c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:15.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.813+0000 7f65b77fe640 1 -- 172.21.15.52:0/2066417596 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f65c8056a50 con 0x7f65d01424f0 2024-05-04T12:19:15.832 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-05-04T12:19:15.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:15.985+0000 7f65d525b640 1 -- 172.21.15.52:0/2066417596 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f65d0075320 con 0x7f65d01424f0 2024-05-04T12:19:16.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.601+0000 7f65b77fe640 1 -- 172.21.15.52:0/2066417596 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) ==== 97+0+0 (secure 0 0 0) 0x7f65c805ad50 con 0x7f65d01424f0 2024-05-04T12:19:16.604 INFO:teuthology.orchestra.run.smithi052.stderr: removed all blocklist entries 2024-05-04T12:19:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.605+0000 7f65d525b640 1 -- 172.21.15.52:0/2066417596 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f65bc0678c0 msgr2=0x7f65bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.605+0000 7f65d525b640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f65bc0678c0 0x7f65bc069d80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f65d01550d0 tx=0x7f65c0008040 comp rx=0 tx=0).stop 2024-05-04T12:19:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.605+0000 7f65d525b640 1 -- 172.21.15.52:0/2066417596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65d01424f0 msgr2=0x7f65d0153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.605+0000 7f65d525b640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65d01424f0 0x7f65d0153e70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f65c800cef0 tx=0x7f65c800a710 comp rx=0 tx=0).stop 2024-05-04T12:19:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.605+0000 7f65d525b640 1 -- 172.21.15.52:0/2066417596 shutdown_connections 2024-05-04T12:19:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.605+0000 7f65d525b640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f65bc0678c0 0x7f65bc069d80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.605+0000 7f65d525b640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65d01424f0 0x7f65d0153e70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.605+0000 7f65d525b640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65d00ceb80 0x7f65d013e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.605+0000 7f65d525b640 1 --2- 172.21.15.52:0/2066417596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f65d0067070 0x7f65d0156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.605+0000 7f65d525b640 1 -- 172.21.15.52:0/2066417596 >> 172.21.15.52:0/2066417596 conn(0x7f65d00b1c10 msgr2=0x7f65d00cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.605+0000 7f65d525b640 1 -- 172.21.15.52:0/2066417596 shutdown_connections 2024-05-04T12:19:16.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.605+0000 7f65d525b640 1 -- 172.21.15.52:0/2066417596 wait complete. 2024-05-04T12:19:16.634 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-04T12:19:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.753+0000 7f331a37e640 1 Processor -- start 2024-05-04T12:19:16.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.753+0000 7f331a37e640 1 -- start start 2024-05-04T12:19:16.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.753+0000 7f331a37e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f331415bf50 0x7f331415c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:16.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.753+0000 7f331a37e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f33140aa0c0 0x7f33140aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:16.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.753+0000 7f331a37e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f331406d1b0 con 0x7f33140ccb50 2024-05-04T12:19:16.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.753+0000 7f331a37e640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f331406d490 con 0x7f33140aa0c0 2024-05-04T12:19:16.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.753+0000 7f331a37e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f331406d770 con 0x7f331415bf50 2024-05-04T12:19:16.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.753+0000 7f33137fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f33140aa0c0 0x7f33140aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:16.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.753+0000 7f33137fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f33140aa0c0 0x7f33140aa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43604/0 (socket says 172.21.15.52:43604) 2024-05-04T12:19:16.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.753+0000 7f33137fe640 1 -- 172.21.15.52:0/1896191296 learned_addr learned my addr 172.21.15.52:0/1896191296 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:16.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.753+0000 7f3313fff640 1 --2- 172.21.15.52:0/1896191296 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f331415bf50 0x7f331415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:16.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f33137fe640 1 -- 172.21.15.52:0/1896191296 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f331415bf50 msgr2=0x7f331415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:16.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f33137fe640 1 --2- 172.21.15.52:0/1896191296 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f331415bf50 0x7f331415c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:16.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f33137fe640 1 -- 172.21.15.52:0/1896191296 >> v1:172.21.15.111:6789/0 conn(0x7f33140ccb50 legacy=0x7f331406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:16.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f33137fe640 1 -- 172.21.15.52:0/1896191296 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331406da50 con 0x7f33140aa0c0 2024-05-04T12:19:16.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f33137fe640 1 --2- 172.21.15.52:0/1896191296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33140aa0c0 0x7f33140aa520 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3308009870 tx=0x7f330801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b04e6b4717ac3b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:16.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f3312ffd640 1 -- 172.21.15.52:0/1896191296 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33080a9610 con 0x7f33140aa0c0 2024-05-04T12:19:16.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f3312ffd640 1 -- 172.21.15.52:0/1896191296 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33080a97b0 con 0x7f33140aa0c0 2024-05-04T12:19:16.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f3312ffd640 1 -- 172.21.15.52:0/1896191296 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33080a9b80 con 0x7f33140aa0c0 2024-05-04T12:19:16.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f331a37e640 1 -- 172.21.15.52:0/1896191296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33140aa0c0 msgr2=0x7f33140aa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:16.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f331a37e640 1 --2- 172.21.15.52:0/1896191296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33140aa0c0 0x7f33140aa520 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3308009870 tx=0x7f330801c530 comp rx=0 tx=0).stop 2024-05-04T12:19:16.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f331a37e640 1 -- 172.21.15.52:0/1896191296 shutdown_connections 2024-05-04T12:19:16.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f331a37e640 1 --2- 172.21.15.52:0/1896191296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33140aa0c0 0x7f33140aa520 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:16.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f331a37e640 1 --2- 172.21.15.52:0/1896191296 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f331415bf50 0x7f331415c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:16.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f331a37e640 1 -- 172.21.15.52:0/1896191296 >> 172.21.15.52:0/1896191296 conn(0x7f33140b14a0 msgr2=0x7f33140b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:16.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f331a37e640 1 -- 172.21.15.52:0/1896191296 shutdown_connections 2024-05-04T12:19:16.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.757+0000 7f331a37e640 1 -- 172.21.15.52:0/1896191296 wait complete. 2024-05-04T12:19:16.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331a37e640 1 Processor -- start 2024-05-04T12:19:16.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331a37e640 1 -- start start 2024-05-04T12:19:16.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331a37e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f33140aa0c0 0x7f331414b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:16.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331a37e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33140ccb50 0x7f3314155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:16.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331a37e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331415bf50 0x7f3314149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:16.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331a37e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f331406cee0 con 0x7f331415bf50 2024-05-04T12:19:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331a37e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f331406cd60 con 0x7f33140ccb50 2024-05-04T12:19:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331a37e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f3314169030 con 0x7f33140aa0c0 2024-05-04T12:19:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331891b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331415bf50 0x7f3314149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331891b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331415bf50 0x7f3314149860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43618/0 (socket says 172.21.15.52:43618) 2024-05-04T12:19:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331891b640 1 -- 172.21.15.52:0/1864574935 learned_addr learned my addr 172.21.15.52:0/1864574935 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f3313fff640 1 --2- 172.21.15.52:0/1864574935 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f33140aa0c0 0x7f331414b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f33137fe640 1 --2- 172.21.15.52:0/1864574935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33140ccb50 0x7f3314155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331891b640 1 -- 172.21.15.52:0/1864574935 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f33140aa0c0 msgr2=0x7f331414b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331891b640 1 --2- 172.21.15.52:0/1864574935 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f33140aa0c0 0x7f331414b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331891b640 1 -- 172.21.15.52:0/1864574935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33140ccb50 msgr2=0x7f3314155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331891b640 1 --2- 172.21.15.52:0/1864574935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33140ccb50 0x7f3314155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331891b640 1 -- 172.21.15.52:0/1864574935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3314155700 con 0x7f331415bf50 2024-05-04T12:19:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331891b640 1 --2- 172.21.15.52:0/1864574935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331415bf50 0x7f3314149860 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f330c00c740 tx=0x7f330c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:16.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f3310ff9640 1 -- 172.21.15.52:0/1864574935 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c016020 con 0x7f331415bf50 2024-05-04T12:19:16.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f3310ff9640 1 -- 172.21.15.52:0/1864574935 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f330c00ce50 con 0x7f331415bf50 2024-05-04T12:19:16.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331a37e640 1 -- 172.21.15.52:0/1864574935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3314148d80 con 0x7f331415bf50 2024-05-04T12:19:16.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f3310ff9640 1 -- 172.21.15.52:0/1864574935 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c003980 con 0x7f331415bf50 2024-05-04T12:19:16.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331a37e640 1 -- 172.21.15.52:0/1864574935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3314149340 con 0x7f331415bf50 2024-05-04T12:19:16.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f331a37e640 1 -- 172.21.15.52:0/1864574935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3314062e50 con 0x7f331415bf50 2024-05-04T12:19:16.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f3310ff9640 1 -- 172.21.15.52:0/1864574935 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f330c003b20 con 0x7f331415bf50 2024-05-04T12:19:16.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.761+0000 7f3310ff9640 1 --2- 172.21.15.52:0/1864574935 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f33000678c0 0x7f3300069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:16.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.765+0000 7f3310ff9640 1 -- 172.21.15.52:0/1864574935 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f330c085650 con 0x7f331415bf50 2024-05-04T12:19:16.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.765+0000 7f3313fff640 1 --2- 172.21.15.52:0/1864574935 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f33000678c0 0x7f3300069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:16.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.765+0000 7f3313fff640 1 --2- 172.21.15.52:0/1864574935 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f33000678c0 0x7f3300069d80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f330400b930 tx=0x7f3304008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:16.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.769+0000 7f3310ff9640 1 -- 172.21.15.52:0/1864574935 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f330c020020 con 0x7f331415bf50 2024-05-04T12:19:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.985+0000 7f331a37e640 1 -- 172.21.15.52:0/1864574935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3314075320 con 0x7f331415bf50 2024-05-04T12:19:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.985+0000 7f3310ff9640 1 -- 172.21.15.52:0/1864574935 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) ==== 80+0+6645 (secure 0 0 0) 0x7f330c0564f0 con 0x7f331415bf50 2024-05-04T12:19:16.990 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:16.990 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-04T12:19:16.990 INFO:teuthology.orchestra.run.smithi052.stdout: "auth_dump": [ 2024-05-04T12:19:16.990 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:16.990 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.a", 2024-05-04T12:19:16.990 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBVJzZmLE53FhAAbGwz3qGhQTZmf2M7JAzqLQ==", 2024-05-04T12:19:16.990 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:16.990 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-04T12:19:16.990 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.b", 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBVJzZmwWHjIhAAHZBv9NPrPIaRLzttDSoPSA==", 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:16.991 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.c", 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBVJzZm9ue4HBAAyNkrNG+mHYXsiFqsZg+WTA==", 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.d", 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBVJzZmB9qbKBAAyijx2yI3L0xfXg/JqR6stg==", 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-04T12:19:16.992 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.0", 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBXJzZmR7w9CRAAk3eqOAw/hgbLNGgR306s8g==", 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.1", 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBaJzZmrX1kDxAAvRHfzSkgWpASb8Z09y5Zdw==", 2024-05-04T12:19:16.993 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:16.994 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-04T12:19:16.994 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-04T12:19:16.994 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:16.994 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:16.994 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:16.994 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:16.994 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.2", 2024-05-04T12:19:16.994 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBdJzZm6CG1FRAAWkuJidO1JSlt5RTWTf0HAg==", 2024-05-04T12:19:16.994 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:16.994 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-04T12:19:16.994 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-04T12:19:16.994 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:16.994 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:16.995 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:16.995 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:16.995 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.3", 2024-05-04T12:19:16.995 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBgJzZmF+TmGxAAla+oLW9K4xDU0n9+tCzYmQ==", 2024-05-04T12:19:16.995 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:16.995 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-04T12:19:16.995 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-04T12:19:16.995 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:16.995 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:16.995 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:16.995 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:16.995 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.4", 2024-05-04T12:19:16.996 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBmJzZm2fBrGRAAwi+07gy0Pcwnq6WIcREE4Q==", 2024-05-04T12:19:16.996 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:16.996 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-04T12:19:16.996 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-04T12:19:16.996 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:16.996 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:16.996 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:16.996 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:16.996 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.5", 2024-05-04T12:19:16.996 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBpJzZmx2WVHxAAyHrMJ70jj7iTXAcme50Lcg==", 2024-05-04T12:19:16.996 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:16.996 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-04T12:19:16.996 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-04T12:19:16.996 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:16.997 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:16.997 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:16.997 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:16.997 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.6", 2024-05-04T12:19:16.997 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBsJzZmq9BuJhAAV5wRyq1ERTydpwTQf4r6aw==", 2024-05-04T12:19:16.997 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:16.997 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-04T12:19:16.997 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-04T12:19:16.997 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:16.997 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:16.997 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:16.997 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:16.997 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.7", 2024-05-04T12:19:16.998 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBvJzZm2pqcLRAA3S4YRX+XDbvL08IlQ8IuLg==", 2024-05-04T12:19:16.998 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:16.998 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-04T12:19:16.998 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-04T12:19:16.998 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:16.998 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:16.998 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:16.998 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:16.998 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.0", 2024-05-04T12:19:16.998 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBVJzZm/x1tLhAApncXD+0NjD0NbekN+kVgdQ==", 2024-05-04T12:19:16.998 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:16.998 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-04T12:19:16.999 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-05-04T12:19:16.999 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-05-04T12:19:16.999 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-05-04T12:19:16.999 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:16.999 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:16.999 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:16.999 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.1", 2024-05-04T12:19:16.999 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBVJzZmLPdXNhAAHRdLfb1zoNaeY/9O9UrEmA==", 2024-05-04T12:19:16.999 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:16.999 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-04T12:19:16.999 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-05-04T12:19:16.999 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-05-04T12:19:16.999 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.2", 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBVJzZmMjeYMxAA1T1RD7vNODSe3lVQLsgokA==", 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.3", 2024-05-04T12:19:17.000 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBWJzZmTHAFABAAW0rbCV51SnDTWE1drZsK+w==", 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.admin", 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBUJzZmagRtORAAnh8L3cPjjG9dyD+slAZ8Ow==", 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow *", 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-04T12:19:17.001 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:17.002 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:17.002 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:17.002 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:17.002 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-mds", 2024-05-04T12:19:17.002 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQB1JzZmoZOyNxAAt9AcXALl+1vq7uT7fRYBWA==", 2024-05-04T12:19:17.002 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:17.002 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-mds" 2024-05-04T12:19:17.002 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:17.002 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:17.002 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:17.003 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-mgr", 2024-05-04T12:19:17.003 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQB1JzZmvK6yNxAAdEvU/rT/Fif8Bk/6V4AmmQ==", 2024-05-04T12:19:17.003 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:17.003 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-04T12:19:17.003 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:17.003 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:17.003 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:17.003 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-osd", 2024-05-04T12:19:17.003 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQB1JzZmCcSyNxAAeWUzAa48e9PpKZAoQudNBw==", 2024-05-04T12:19:17.003 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:17.003 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-osd" 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-rbd", 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQB1JzZmeNmyNxAApzM5965/1uwsMBroT3IM7g==", 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQB1JzZmkO+yNxAA/KThnGUR42m4bJlc9rRJxQ==", 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-04T12:19:17.004 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:17.005 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:17.005 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:17.005 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-rgw", 2024-05-04T12:19:17.005 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQB1JzZmdwazNxAA3JSc4krzCZFRUYE5ZtOLwA==", 2024-05-04T12:19:17.005 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:17.005 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-04T12:19:17.005 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:17.006 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:17.006 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:17.006 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mgr.x", 2024-05-04T12:19:17.006 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBVJzZmRRC8ERAAbWLMj3635LsYGHfICJ1CDQ==", 2024-05-04T12:19:17.006 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:17.007 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow *", 2024-05-04T12:19:17.007 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile mgr", 2024-05-04T12:19:17.007 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:17.007 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:17.007 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:17.007 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:17.007 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mgr.y", 2024-05-04T12:19:17.008 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBVJzZmXy4GDxAA6TDE57YwmNNvp2Wmkni8OQ==", 2024-05-04T12:19:17.008 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-04T12:19:17.008 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow *", 2024-05-04T12:19:17.008 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile mgr", 2024-05-04T12:19:17.008 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-04T12:19:17.008 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:17.008 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:17.008 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-05-04T12:19:17.008 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-04T12:19:17.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.989+0000 7f331a37e640 1 -- 172.21.15.52:0/1864574935 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f33000678c0 msgr2=0x7f3300069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.989+0000 7f331a37e640 1 --2- 172.21.15.52:0/1864574935 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f33000678c0 0x7f3300069d80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f330400b930 tx=0x7f3304008040 comp rx=0 tx=0).stop 2024-05-04T12:19:17.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.989+0000 7f331a37e640 1 -- 172.21.15.52:0/1864574935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331415bf50 msgr2=0x7f3314149860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.989+0000 7f331a37e640 1 --2- 172.21.15.52:0/1864574935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331415bf50 0x7f3314149860 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f330c00c740 tx=0x7f330c00cc00 comp rx=0 tx=0).stop 2024-05-04T12:19:17.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.989+0000 7f331a37e640 1 -- 172.21.15.52:0/1864574935 shutdown_connections 2024-05-04T12:19:17.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.989+0000 7f331a37e640 1 --2- 172.21.15.52:0/1864574935 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f33000678c0 0x7f3300069d80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.989+0000 7f331a37e640 1 --2- 172.21.15.52:0/1864574935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331415bf50 0x7f3314149860 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.989+0000 7f331a37e640 1 --2- 172.21.15.52:0/1864574935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33140ccb50 0x7f3314155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.989+0000 7f331a37e640 1 --2- 172.21.15.52:0/1864574935 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f33140aa0c0 0x7f331414b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.989+0000 7f331a37e640 1 -- 172.21.15.52:0/1864574935 >> 172.21.15.52:0/1864574935 conn(0x7f33140b14a0 msgr2=0x7f33140a9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:17.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.989+0000 7f331a37e640 1 -- 172.21.15.52:0/1864574935 shutdown_connections 2024-05-04T12:19:17.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:16.989+0000 7f331a37e640 1 -- 172.21.15.52:0/1864574935 wait complete. 2024-05-04T12:19:17.011 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-04T12:19:17.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.129+0000 7fec73a1f640 1 Processor -- start 2024-05-04T12:19:17.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.129+0000 7fec73a1f640 1 -- start start 2024-05-04T12:19:17.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.129+0000 7fec73a1f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fec6c0ccdb0 0x7fec6c0723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.129+0000 7fec73a1f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fec6c15c7f0 0x7fec6c15c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.129+0000 7fec73a1f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec6c072aa0 con 0x7fec6c0cd190 2024-05-04T12:19:17.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.129+0000 7fec73a1f640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fec6c070690 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.129+0000 7fec73a1f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fec6c0708e0 con 0x7fec6c15c7f0 2024-05-04T12:19:17.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.129+0000 7fec717bb640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fec6c0ccdb0 0x7fec6c0723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.129+0000 7fec717bb640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fec6c0ccdb0 0x7fec6c0723e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43620/0 (socket says 172.21.15.52:43620) 2024-05-04T12:19:17.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.129+0000 7fec717bb640 1 -- 172.21.15.52:0/2522055592 learned_addr learned my addr 172.21.15.52:0/2522055592 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:17.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec70fba640 1 --2- 172.21.15.52:0/2522055592 >> v2:172.21.15.111:3300/0 conn(0x7fec6c15c7f0 0x7fec6c15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec717bb640 1 -- 172.21.15.52:0/2522055592 >> v2:172.21.15.111:3300/0 conn(0x7fec6c15c7f0 msgr2=0x7fec6c15c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:17.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec717bb640 1 --2- 172.21.15.52:0/2522055592 >> v2:172.21.15.111:3300/0 conn(0x7fec6c15c7f0 0x7fec6c15c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec717bb640 1 -- 172.21.15.52:0/2522055592 >> v1:172.21.15.111:6789/0 conn(0x7fec6c0cd190 legacy=0x7fec6c158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec717bb640 1 -- 172.21.15.52:0/2522055592 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6c070bc0 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec717bb640 1 --2- 172.21.15.52:0/2522055592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec6c0ccdb0 0x7fec6c0723e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fec6c072d80 tx=0x7fec6801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4186f5bd02add6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:17.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec63fff640 1 -- 172.21.15.52:0/2522055592 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec680ac070 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec63fff640 1 -- 172.21.15.52:0/2522055592 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec680a6440 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec63fff640 1 -- 172.21.15.52:0/2522055592 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec680a6810 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec73a1f640 1 -- 172.21.15.52:0/2522055592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec6c0ccdb0 msgr2=0x7fec6c0723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec73a1f640 1 --2- 172.21.15.52:0/2522055592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec6c0ccdb0 0x7fec6c0723e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fec6c072d80 tx=0x7fec6801c960 comp rx=0 tx=0).stop 2024-05-04T12:19:17.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec73a1f640 1 -- 172.21.15.52:0/2522055592 shutdown_connections 2024-05-04T12:19:17.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec73a1f640 1 --2- 172.21.15.52:0/2522055592 >> v2:172.21.15.111:3300/0 conn(0x7fec6c15c7f0 0x7fec6c15c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec73a1f640 1 --2- 172.21.15.52:0/2522055592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec6c0ccdb0 0x7fec6c0723e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.133+0000 7fec73a1f640 1 -- 172.21.15.52:0/2522055592 >> 172.21.15.52:0/2522055592 conn(0x7fec6c0b14a0 msgr2=0x7fec6c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec73a1f640 1 -- 172.21.15.52:0/2522055592 shutdown_connections 2024-05-04T12:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec73a1f640 1 -- 172.21.15.52:0/2522055592 wait complete. 2024-05-04T12:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec73a1f640 1 Processor -- start 2024-05-04T12:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec73a1f640 1 -- start start 2024-05-04T12:19:17.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec73a1f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec6c0ccdb0 0x7fec6c14f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec73a1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec6c0cd190 0x7fec6c159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec73a1f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec6c15c7f0 0x7fec6c0b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec73a1f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c168a10 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec73a1f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c168890 con 0x7fec6c0cd190 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec73a1f640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fec6c168b90 con 0x7fec6c15c7f0 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec717bb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec6c0ccdb0 0x7fec6c14f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec717bb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec6c0ccdb0 0x7fec6c14f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43624/0 (socket says 172.21.15.52:43624) 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec717bb640 1 -- 172.21.15.52:0/1831514583 learned_addr learned my addr 172.21.15.52:0/1831514583 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec70fba640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec6c0cd190 0x7fec6c159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec71fbc640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec6c15c7f0 0x7fec6c0b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec717bb640 1 -- 172.21.15.52:0/1831514583 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec6c15c7f0 msgr2=0x7fec6c0b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec717bb640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec6c15c7f0 0x7fec6c0b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec717bb640 1 -- 172.21.15.52:0/1831514583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec6c0cd190 msgr2=0x7fec6c159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec717bb640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec6c0cd190 0x7fec6c159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec717bb640 1 -- 172.21.15.52:0/1831514583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6c15a1f0 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec70fba640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec6c0cd190 0x7fec6c159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:17.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec717bb640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec6c0ccdb0 0x7fec6c14f4d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fec68019570 tx=0x7fec68019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:17.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec61ffb640 1 -- 172.21.15.52:0/1831514583 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec680ac040 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec61ffb640 1 -- 172.21.15.52:0/1831514583 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec680bc450 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.137+0000 7fec61ffb640 1 -- 172.21.15.52:0/1831514583 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec680bc840 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.141+0000 7fec73a1f640 1 -- 172.21.15.52:0/1831514583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6c159950 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.141+0000 7fec73a1f640 1 -- 172.21.15.52:0/1831514583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec6c159f10 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.141+0000 7fec73a1f640 1 -- 172.21.15.52:0/1831514583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec6c062e50 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.141+0000 7fec61ffb640 1 -- 172.21.15.52:0/1831514583 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fec680af070 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.141+0000 7fec61ffb640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fec500678c0 0x7fec50069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.141+0000 7fec61ffb640 1 -- 172.21.15.52:0/1831514583 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fec6811fec0 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.141+0000 7fec70fba640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fec500678c0 0x7fec50069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.141+0000 7fec70fba640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fec500678c0 0x7fec50069d80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec6c1583c0 tx=0x7fec5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:17.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.145+0000 7fec61ffb640 1 -- 172.21.15.52:0/1831514583 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fec680165e0 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.365+0000 7fec73a1f640 1 -- 172.21.15.52:0/1831514583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fec6c075320 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.369+0000 7fec61ffb640 1 -- 172.21.15.52:0/1831514583 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fec680f0cb0 con 0x7fec6c0ccdb0 2024-05-04T12:19:17.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.373+0000 7fec73a1f640 1 -- 172.21.15.52:0/1831514583 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fec500678c0 msgr2=0x7fec50069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.373+0000 7fec73a1f640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fec500678c0 0x7fec50069d80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec6c1583c0 tx=0x7fec5c008040 comp rx=0 tx=0).stop 2024-05-04T12:19:17.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.373+0000 7fec73a1f640 1 -- 172.21.15.52:0/1831514583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec6c0ccdb0 msgr2=0x7fec6c14f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.373+0000 7fec73a1f640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec6c0ccdb0 0x7fec6c14f4d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fec68019570 tx=0x7fec68019700 comp rx=0 tx=0).stop 2024-05-04T12:19:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.373+0000 7fec73a1f640 1 -- 172.21.15.52:0/1831514583 shutdown_connections 2024-05-04T12:19:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.373+0000 7fec73a1f640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fec500678c0 0x7fec50069d80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.373+0000 7fec73a1f640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec6c15c7f0 0x7fec6c0b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.373+0000 7fec73a1f640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec6c0cd190 0x7fec6c159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.373+0000 7fec73a1f640 1 --2- 172.21.15.52:0/1831514583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec6c0ccdb0 0x7fec6c14f4d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.373+0000 7fec73a1f640 1 -- 172.21.15.52:0/1831514583 >> 172.21.15.52:0/1831514583 conn(0x7fec6c0b14a0 msgr2=0x7fec6c0cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.373+0000 7fec73a1f640 1 -- 172.21.15.52:0/1831514583 shutdown_connections 2024-05-04T12:19:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.373+0000 7fec73a1f640 1 -- 172.21.15.52:0/1831514583 wait complete. 2024-05-04T12:19:17.394 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-04T12:19:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.509+0000 7f4ca6cce640 1 Processor -- start 2024-05-04T12:19:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.509+0000 7f4ca6cce640 1 -- start start 2024-05-04T12:19:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.509+0000 7f4ca6cce640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4ca0071900 0x7f4ca0071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.509+0000 7f4ca6cce640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4ca00aa710 0x7f4ca00aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.509+0000 7f4ca6cce640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca015e8c0 con 0x7f4ca00adfa0 2024-05-04T12:19:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.509+0000 7f4ca6cce640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca0141650 con 0x7f4ca00aa710 2024-05-04T12:19:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.509+0000 7f4ca6cce640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca015f470 con 0x7f4ca0071900 2024-05-04T12:19:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.509+0000 7f4c9ffff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4ca00aa710 0x7f4ca00aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.509+0000 7f4ca526b640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f4ca00adfa0 0x7f4ca00ae3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:37720/0 (socket says 172.21.15.52:37720) 2024-05-04T12:19:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.509+0000 7f4ca526b640 1 -- 172.21.15.52:0/2062573646 learned_addr learned my addr 172.21.15.52:0/2062573646 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.509+0000 7f4ca4a6a640 1 --2- 172.21.15.52:0/2062573646 >> v2:172.21.15.111:3300/0 conn(0x7f4ca0071900 0x7f4ca0071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.509+0000 7f4c9f7fe640 1 -- 172.21.15.52:0/2062573646 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 344441421 0 0) 0x7f4c98001120 con 0x7f4ca00adfa0 2024-05-04T12:19:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.509+0000 7f4c9f7fe640 1 -- 172.21.15.52:0/2062573646 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c88003530 con 0x7f4ca00adfa0 2024-05-04T12:19:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4c9ffff640 1 -- 172.21.15.52:0/2062573646 >> v2:172.21.15.111:3300/0 conn(0x7f4ca0071900 msgr2=0x7f4ca0071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4c9ffff640 1 --2- 172.21.15.52:0/2062573646 >> v2:172.21.15.111:3300/0 conn(0x7f4ca0071900 0x7f4ca0071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4c9ffff640 1 -- 172.21.15.52:0/2062573646 >> v1:172.21.15.52:6789/0 conn(0x7f4ca00adfa0 legacy=0x7f4ca00ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4c9ffff640 1 -- 172.21.15.52:0/2062573646 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca015eb40 con 0x7f4ca00aa710 2024-05-04T12:19:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4c9ffff640 1 --2- 172.21.15.52:0/2062573646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca00aa710 0x7f4ca00aabc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00d1d10 tx=0x7f4c9401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764d78f2d74b10d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4c9f7fe640 1 -- 172.21.15.52:0/2062573646 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c940ab5f0 con 0x7f4ca00aa710 2024-05-04T12:19:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4c9f7fe640 1 -- 172.21.15.52:0/2062573646 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c940b0070 con 0x7f4ca00aa710 2024-05-04T12:19:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4c9f7fe640 1 -- 172.21.15.52:0/2062573646 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c940ab990 con 0x7f4ca00aa710 2024-05-04T12:19:17.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4ca6cce640 1 -- 172.21.15.52:0/2062573646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca00aa710 msgr2=0x7f4ca00aabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4ca6cce640 1 --2- 172.21.15.52:0/2062573646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca00aa710 0x7f4ca00aabc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00d1d10 tx=0x7f4c9401c3f0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4ca6cce640 1 -- 172.21.15.52:0/2062573646 shutdown_connections 2024-05-04T12:19:17.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4ca6cce640 1 --2- 172.21.15.52:0/2062573646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca00aa710 0x7f4ca00aabc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4ca6cce640 1 --2- 172.21.15.52:0/2062573646 >> v2:172.21.15.111:3300/0 conn(0x7f4ca0071900 0x7f4ca0071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4ca6cce640 1 -- 172.21.15.52:0/2062573646 >> 172.21.15.52:0/2062573646 conn(0x7f4ca00d0ff0 msgr2=0x7f4ca00d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:17.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4ca6cce640 1 -- 172.21.15.52:0/2062573646 shutdown_connections 2024-05-04T12:19:17.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.513+0000 7f4ca6cce640 1 -- 172.21.15.52:0/2062573646 wait complete. 2024-05-04T12:19:17.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca6cce640 1 Processor -- start 2024-05-04T12:19:17.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca6cce640 1 -- start start 2024-05-04T12:19:17.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca6cce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca0071900 0x7f4ca01547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca6cce640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ca00aa710 0x7f4ca013b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca6cce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca00adfa0 0x7f4ca013b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca6cce640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca00aec50 con 0x7f4ca00adfa0 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca6cce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca00aead0 con 0x7f4ca0071900 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca6cce640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca01428e0 con 0x7f4ca00aa710 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca526b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca00adfa0 0x7f4ca013b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca526b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca00adfa0 0x7f4ca013b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43638/0 (socket says 172.21.15.52:43638) 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca526b640 1 -- 172.21.15.52:0/1628522528 learned_addr learned my addr 172.21.15.52:0/1628522528 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca4a6a640 1 --2- 172.21.15.52:0/1628522528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca0071900 0x7f4ca01547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4c9ffff640 1 --2- 172.21.15.52:0/1628522528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ca00aa710 0x7f4ca013b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca526b640 1 -- 172.21.15.52:0/1628522528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ca00aa710 msgr2=0x7f4ca013b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca526b640 1 --2- 172.21.15.52:0/1628522528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ca00aa710 0x7f4ca013b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca526b640 1 -- 172.21.15.52:0/1628522528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca0071900 msgr2=0x7f4ca01547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca526b640 1 --2- 172.21.15.52:0/1628522528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca0071900 0x7f4ca01547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca526b640 1 -- 172.21.15.52:0/1628522528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca013cd90 con 0x7f4ca00adfa0 2024-05-04T12:19:17.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca526b640 1 --2- 172.21.15.52:0/1628522528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca00adfa0 0x7f4ca013b600 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c3f0 tx=0x7f4c9800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:17.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4c9d7fa640 1 -- 172.21.15.52:0/1628522528 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98017070 con 0x7f4ca00adfa0 2024-05-04T12:19:17.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4c9d7fa640 1 -- 172.21.15.52:0/1628522528 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c98005cf0 con 0x7f4ca00adfa0 2024-05-04T12:19:17.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca6cce640 1 -- 172.21.15.52:0/1628522528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca013d070 con 0x7f4ca00adfa0 2024-05-04T12:19:17.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4c9d7fa640 1 -- 172.21.15.52:0/1628522528 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c980072f0 con 0x7f4ca00adfa0 2024-05-04T12:19:17.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.517+0000 7f4ca6cce640 1 -- 172.21.15.52:0/1628522528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca013d630 con 0x7f4ca00adfa0 2024-05-04T12:19:17.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.521+0000 7f4ca6cce640 1 -- 172.21.15.52:0/1628522528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca0062e50 con 0x7f4ca00adfa0 2024-05-04T12:19:17.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.521+0000 7f4c9d7fa640 1 -- 172.21.15.52:0/1628522528 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4c9801a070 con 0x7f4ca00adfa0 2024-05-04T12:19:17.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.521+0000 7f4c9d7fa640 1 --2- 172.21.15.52:0/1628522528 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4c8c0678c0 0x7f4c8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.521+0000 7f4c9d7fa640 1 -- 172.21.15.52:0/1628522528 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f4c98084c80 con 0x7f4ca00adfa0 2024-05-04T12:19:17.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.521+0000 7f4ca4a6a640 1 --2- 172.21.15.52:0/1628522528 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4c8c0678c0 0x7f4c8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.521+0000 7f4ca4a6a640 1 --2- 172.21.15.52:0/1628522528 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4c8c0678c0 0x7f4c8c069d80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c90003fd0 tx=0x7f4c900039a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:17.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.525+0000 7f4c9d7fa640 1 -- 172.21.15.52:0/1628522528 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4ca015e8c0 con 0x7f4ca00adfa0 2024-05-04T12:19:17.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.749+0000 7f4ca6cce640 1 -- 172.21.15.52:0/1628522528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f4ca00ab850 con 0x7f4ca00adfa0 2024-05-04T12:19:17.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.753+0000 7f4c9d7fa640 1 -- 172.21.15.52:0/1628522528 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f4c98055ba0 con 0x7f4ca00adfa0 2024-05-04T12:19:17.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.757+0000 7f4ca6cce640 1 -- 172.21.15.52:0/1628522528 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4c8c0678c0 msgr2=0x7f4c8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.757+0000 7f4ca6cce640 1 --2- 172.21.15.52:0/1628522528 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4c8c0678c0 0x7f4c8c069d80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c90003fd0 tx=0x7f4c900039a0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.757+0000 7f4ca6cce640 1 -- 172.21.15.52:0/1628522528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca00adfa0 msgr2=0x7f4ca013b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.757+0000 7f4ca6cce640 1 --2- 172.21.15.52:0/1628522528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca00adfa0 0x7f4ca013b600 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c3f0 tx=0x7f4c9800c8b0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.757+0000 7f4ca6cce640 1 -- 172.21.15.52:0/1628522528 shutdown_connections 2024-05-04T12:19:17.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.757+0000 7f4ca6cce640 1 --2- 172.21.15.52:0/1628522528 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4c8c0678c0 0x7f4c8c069d80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.757+0000 7f4ca6cce640 1 --2- 172.21.15.52:0/1628522528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca00adfa0 0x7f4ca013b600 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.757+0000 7f4ca6cce640 1 --2- 172.21.15.52:0/1628522528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ca00aa710 0x7f4ca013b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.757+0000 7f4ca6cce640 1 --2- 172.21.15.52:0/1628522528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca0071900 0x7f4ca01547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.757+0000 7f4ca6cce640 1 -- 172.21.15.52:0/1628522528 >> 172.21.15.52:0/1628522528 conn(0x7f4ca00d0ff0 msgr2=0x7f4ca00ae380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:17.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.757+0000 7f4ca6cce640 1 -- 172.21.15.52:0/1628522528 shutdown_connections 2024-05-04T12:19:17.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.757+0000 7f4ca6cce640 1 -- 172.21.15.52:0/1628522528 wait complete. 2024-05-04T12:19:17.778 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-04T12:19:17.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a3876640 1 Processor -- start 2024-05-04T12:19:17.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a3876640 1 -- start start 2024-05-04T12:19:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a3876640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f119c15bf50 0x7f119c15c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a3876640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f119c0aa0c0 0x7f119c0aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a3876640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f119c06d1b0 con 0x7f119c0ccb50 2024-05-04T12:19:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a3876640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f119c06d490 con 0x7f119c0aa0c0 2024-05-04T12:19:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a3876640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f119c06d770 con 0x7f119c15bf50 2024-05-04T12:19:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a0e11640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f119c0aa0c0 0x7f119c0aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a0e11640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f119c0aa0c0 0x7f119c0aa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43648/0 (socket says 172.21.15.52:43648) 2024-05-04T12:19:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a0e11640 1 -- 172.21.15.52:0/3935200883 learned_addr learned my addr 172.21.15.52:0/3935200883 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a1612640 1 --2- 172.21.15.52:0/3935200883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f119c15bf50 0x7f119c15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a0e11640 1 -- 172.21.15.52:0/3935200883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f119c15bf50 msgr2=0x7f119c15c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a0e11640 1 --2- 172.21.15.52:0/3935200883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f119c15bf50 0x7f119c15c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a0e11640 1 -- 172.21.15.52:0/3935200883 >> v1:172.21.15.111:6789/0 conn(0x7f119c0ccb50 legacy=0x7f119c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a0e11640 1 -- 172.21.15.52:0/3935200883 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119c06da50 con 0x7f119c0aa0c0 2024-05-04T12:19:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.893+0000 7f11a1612640 1 --2- 172.21.15.52:0/3935200883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f119c15bf50 0x7f119c15c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:17.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a0e11640 1 --2- 172.21.15.52:0/3935200883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c0aa0c0 0x7f119c0aa520 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f118c009870 tx=0x7f118c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d21e159227e36 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:17.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f1193fff640 1 -- 172.21.15.52:0/3935200883 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c0a7070 con 0x7f119c0aa0c0 2024-05-04T12:19:17.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f1193fff640 1 -- 172.21.15.52:0/3935200883 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118c0b0070 con 0x7f119c0aa0c0 2024-05-04T12:19:17.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f1193fff640 1 -- 172.21.15.52:0/3935200883 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c0ab890 con 0x7f119c0aa0c0 2024-05-04T12:19:17.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 -- 172.21.15.52:0/3935200883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c0aa0c0 msgr2=0x7f119c0aa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 --2- 172.21.15.52:0/3935200883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c0aa0c0 0x7f119c0aa520 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f118c009870 tx=0x7f118c01c4f0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 -- 172.21.15.52:0/3935200883 shutdown_connections 2024-05-04T12:19:17.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 --2- 172.21.15.52:0/3935200883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c0aa0c0 0x7f119c0aa520 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 --2- 172.21.15.52:0/3935200883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f119c15bf50 0x7f119c15c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 -- 172.21.15.52:0/3935200883 >> 172.21.15.52:0/3935200883 conn(0x7f119c0b14a0 msgr2=0x7f119c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:17.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 -- 172.21.15.52:0/3935200883 shutdown_connections 2024-05-04T12:19:17.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 -- 172.21.15.52:0/3935200883 wait complete. 2024-05-04T12:19:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 Processor -- start 2024-05-04T12:19:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 -- start start 2024-05-04T12:19:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c0aa0c0 0x7f119c1478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f119c0ccb50 0x7f119c1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f119c15bf50 0x7f119c1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f119c06cee0 con 0x7f119c0aa0c0 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f119c06cd60 con 0x7f119c15bf50 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a3876640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f119c169030 con 0x7f119c0ccb50 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a1612640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c0aa0c0 0x7f119c1478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a1612640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c0aa0c0 0x7f119c1478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43660/0 (socket says 172.21.15.52:43660) 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.897+0000 7f11a1612640 1 -- 172.21.15.52:0/3843428173 learned_addr learned my addr 172.21.15.52:0/3843428173 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f11a1612640 1 -- 172.21.15.52:0/3843428173 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f119c0ccb50 msgr2=0x7f119c1576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f11a0e11640 1 --2- 172.21.15.52:0/3843428173 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f119c0ccb50 0x7f119c1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f11a1e13640 1 --2- 172.21.15.52:0/3843428173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f119c15bf50 0x7f119c1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f11a1612640 1 --2- 172.21.15.52:0/3843428173 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f119c0ccb50 0x7f119c1576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f11a1612640 1 -- 172.21.15.52:0/3843428173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f119c15bf50 msgr2=0x7f119c1537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f11a1612640 1 --2- 172.21.15.52:0/3843428173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f119c15bf50 0x7f119c1537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f11a1612640 1 -- 172.21.15.52:0/3843428173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119c156070 con 0x7f119c0aa0c0 2024-05-04T12:19:17.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f11a1612640 1 --2- 172.21.15.52:0/3843428173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c0aa0c0 0x7f119c1478e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f119800ce00 tx=0x7f119800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:17.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f1191ffb640 1 -- 172.21.15.52:0/3843428173 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119800f890 con 0x7f119c0aa0c0 2024-05-04T12:19:17.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f1191ffb640 1 -- 172.21.15.52:0/3843428173 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119800fa30 con 0x7f119c0aa0c0 2024-05-04T12:19:17.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f11a3876640 1 -- 172.21.15.52:0/3843428173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119c156350 con 0x7f119c0aa0c0 2024-05-04T12:19:17.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f1191ffb640 1 -- 172.21.15.52:0/3843428173 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119800fe00 con 0x7f119c0aa0c0 2024-05-04T12:19:17.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f11a3876640 1 -- 172.21.15.52:0/3843428173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f119c156910 con 0x7f119c0aa0c0 2024-05-04T12:19:17.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f11a3876640 1 -- 172.21.15.52:0/3843428173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119c062e50 con 0x7f119c0aa0c0 2024-05-04T12:19:17.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f1191ffb640 1 -- 172.21.15.52:0/3843428173 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1198018400 con 0x7f119c0aa0c0 2024-05-04T12:19:17.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f1191ffb640 1 --2- 172.21.15.52:0/3843428173 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f11800678c0 0x7f1180069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:17.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f1191ffb640 1 -- 172.21.15.52:0/3843428173 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f11980867b0 con 0x7f119c0aa0c0 2024-05-04T12:19:17.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f11a0e11640 1 --2- 172.21.15.52:0/3843428173 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f11800678c0 0x7f1180069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:17.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.901+0000 7f11a0e11640 1 --2- 172.21.15.52:0/3843428173 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f11800678c0 0x7f1180069d80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f118c0af000 tx=0x7f118c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:17.905+0000 7f1191ffb640 1 -- 172.21.15.52:0/3843428173 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1198014030 con 0x7f119c0aa0c0 2024-05-04T12:19:18.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.129+0000 7f11a3876640 1 -- 172.21.15.52:0/3843428173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f119c072b60 con 0x7f119c0aa0c0 2024-05-04T12:19:18.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.133+0000 7f1191ffb640 1 -- 172.21.15.52:0/3843428173 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7f119800ae20 con 0x7f119c0aa0c0 2024-05-04T12:19:18.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.137+0000 7f11a3876640 1 -- 172.21.15.52:0/3843428173 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f11800678c0 msgr2=0x7f1180069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.137+0000 7f11a3876640 1 --2- 172.21.15.52:0/3843428173 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f11800678c0 0x7f1180069d80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f118c0af000 tx=0x7f118c008040 comp rx=0 tx=0).stop 2024-05-04T12:19:18.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.137+0000 7f11a3876640 1 -- 172.21.15.52:0/3843428173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c0aa0c0 msgr2=0x7f119c1478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.137+0000 7f11a3876640 1 --2- 172.21.15.52:0/3843428173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c0aa0c0 0x7f119c1478e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f119800ce00 tx=0x7f119800a650 comp rx=0 tx=0).stop 2024-05-04T12:19:18.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.137+0000 7f11a3876640 1 -- 172.21.15.52:0/3843428173 shutdown_connections 2024-05-04T12:19:18.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.137+0000 7f11a3876640 1 --2- 172.21.15.52:0/3843428173 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f11800678c0 0x7f1180069d80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.137+0000 7f11a3876640 1 --2- 172.21.15.52:0/3843428173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f119c15bf50 0x7f119c1537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.137+0000 7f11a3876640 1 --2- 172.21.15.52:0/3843428173 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f119c0ccb50 0x7f119c1576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.137+0000 7f11a3876640 1 --2- 172.21.15.52:0/3843428173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c0aa0c0 0x7f119c1478e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.137+0000 7f11a3876640 1 -- 172.21.15.52:0/3843428173 >> 172.21.15.52:0/3843428173 conn(0x7f119c0b14a0 msgr2=0x7f119c1586c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:18.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.137+0000 7f11a3876640 1 -- 172.21.15.52:0/3843428173 shutdown_connections 2024-05-04T12:19:18.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.137+0000 7f11a3876640 1 -- 172.21.15.52:0/3843428173 wait complete. 2024-05-04T12:19:18.157 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-04T12:19:18.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.281+0000 7fc5e963c640 1 Processor -- start 2024-05-04T12:19:18.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.281+0000 7fc5e963c640 1 -- start start 2024-05-04T12:19:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e963c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc5e4161de0 0x7fc5e4162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e963c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc5e40c7f80 0x7fc5e40c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e963c640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5e40cf4c0 con 0x7fc5e40706d0 2024-05-04T12:19:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e963c640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc5e40cdbf0 con 0x7fc5e4161de0 2024-05-04T12:19:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e963c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fc5e40cded0 con 0x7fc5e40c7f80 2024-05-04T12:19:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e27fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc5e4161de0 0x7fc5e4162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e27fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc5e4161de0 0x7fc5e4162440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43668/0 (socket says 172.21.15.52:43668) 2024-05-04T12:19:18.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e27fc640 1 -- 172.21.15.52:0/701797993 learned_addr learned my addr 172.21.15.52:0/701797993 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:18.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e37fe640 1 --2- 172.21.15.52:0/701797993 >> v2:172.21.15.111:3300/0 conn(0x7fc5e40c7f80 0x7fc5e40c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:18.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e27fc640 1 -- 172.21.15.52:0/701797993 >> v2:172.21.15.111:3300/0 conn(0x7fc5e40c7f80 msgr2=0x7fc5e40c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:18.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e27fc640 1 --2- 172.21.15.52:0/701797993 >> v2:172.21.15.111:3300/0 conn(0x7fc5e40c7f80 0x7fc5e40c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e27fc640 1 -- 172.21.15.52:0/701797993 >> v1:172.21.15.52:6789/0 conn(0x7fc5e40706d0 legacy=0x7fc5e4070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e27fc640 1 -- 172.21.15.52:0/701797993 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e40ce1b0 con 0x7fc5e4161de0 2024-05-04T12:19:18.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e1ffb640 1 -- 172.21.15.52:0/701797993 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 837941575 0 0) 0x7fc5d40014a0 con 0x7fc5e40706d0 2024-05-04T12:19:18.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e37fe640 1 --2- 172.21.15.52:0/701797993 >> v2:172.21.15.111:3300/0 conn(0x7fc5e40c7f80 0x7fc5e40c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:18.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e27fc640 1 --2- 172.21.15.52:0/701797993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e4161de0 0x7fc5e4162440 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40b3520 tx=0x7fc5d801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aee84c296e2e91e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:18.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e1ffb640 1 -- 172.21.15.52:0/701797993 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80ab660 con 0x7fc5e4161de0 2024-05-04T12:19:18.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e1ffb640 1 -- 172.21.15.52:0/701797993 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5d80b0070 con 0x7fc5e4161de0 2024-05-04T12:19:18.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.285+0000 7fc5e1ffb640 1 -- 172.21.15.52:0/701797993 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80aba20 con 0x7fc5e4161de0 2024-05-04T12:19:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 -- 172.21.15.52:0/701797993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e4161de0 msgr2=0x7fc5e4162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 --2- 172.21.15.52:0/701797993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e4161de0 0x7fc5e4162440 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40b3520 tx=0x7fc5d801c460 comp rx=0 tx=0).stop 2024-05-04T12:19:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 -- 172.21.15.52:0/701797993 shutdown_connections 2024-05-04T12:19:18.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 --2- 172.21.15.52:0/701797993 >> v2:172.21.15.111:3300/0 conn(0x7fc5e40c7f80 0x7fc5e40c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 --2- 172.21.15.52:0/701797993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e4161de0 0x7fc5e4162440 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 -- 172.21.15.52:0/701797993 >> 172.21.15.52:0/701797993 conn(0x7fc5e4067070 msgr2=0x7fc5e4067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:18.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 -- 172.21.15.52:0/701797993 shutdown_connections 2024-05-04T12:19:18.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 -- 172.21.15.52:0/701797993 wait complete. 2024-05-04T12:19:18.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 Processor -- start 2024-05-04T12:19:18.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 -- start start 2024-05-04T12:19:18.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e40706d0 0x7fc5e4139720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:18.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e40c7f80 0x7fc5e41fa230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:18.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5e41f8620 0x7fc5e41f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:18.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e4162b00 con 0x7fc5e40c7f80 2024-05-04T12:19:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e4162980 con 0x7fc5e40706d0 2024-05-04T12:19:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e416b060 con 0x7fc5e41f8620 2024-05-04T12:19:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e27fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e40c7f80 0x7fc5e41fa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e27fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e40c7f80 0x7fc5e41fa230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43678/0 (socket says 172.21.15.52:43678) 2024-05-04T12:19:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e27fc640 1 -- 172.21.15.52:0/3206780530 learned_addr learned my addr 172.21.15.52:0/3206780530 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e2ffd640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e40706d0 0x7fc5e4139720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e37fe640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5e41f8620 0x7fc5e41f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e27fc640 1 -- 172.21.15.52:0/3206780530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5e41f8620 msgr2=0x7fc5e41f8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e27fc640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5e41f8620 0x7fc5e41f8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e27fc640 1 -- 172.21.15.52:0/3206780530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e40706d0 msgr2=0x7fc5e4139720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e27fc640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e40706d0 0x7fc5e4139720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e27fc640 1 -- 172.21.15.52:0/3206780530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e415d470 con 0x7fc5e40c7f80 2024-05-04T12:19:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e2ffd640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e40706d0 0x7fc5e4139720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e27fc640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e40c7f80 0x7fc5e41fa230 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8019070 tx=0x7fc5d801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:18.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5cb7fe640 1 -- 172.21.15.52:0/3206780530 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80b4800 con 0x7fc5e40c7f80 2024-05-04T12:19:18.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5cb7fe640 1 -- 172.21.15.52:0/3206780530 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5d80b49a0 con 0x7fc5e40c7f80 2024-05-04T12:19:18.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 -- 172.21.15.52:0/3206780530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e4157820 con 0x7fc5e40c7f80 2024-05-04T12:19:18.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5cb7fe640 1 -- 172.21.15.52:0/3206780530 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80b4d90 con 0x7fc5e40c7f80 2024-05-04T12:19:18.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.289+0000 7fc5e963c640 1 -- 172.21.15.52:0/3206780530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e41f9470 con 0x7fc5e40c7f80 2024-05-04T12:19:18.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.293+0000 7fc5e963c640 1 -- 172.21.15.52:0/3206780530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e4062e50 con 0x7fc5e40c7f80 2024-05-04T12:19:18.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.293+0000 7fc5cb7fe640 1 -- 172.21.15.52:0/3206780530 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc5d80c7440 con 0x7fc5e40c7f80 2024-05-04T12:19:18.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.293+0000 7fc5cb7fe640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5d00678c0 0x7fc5d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:18.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.293+0000 7fc5e2ffd640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5d00678c0 0x7fc5d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:18.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.293+0000 7fc5cb7fe640 1 -- 172.21.15.52:0/3206780530 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fc5d8120230 con 0x7fc5e40c7f80 2024-05-04T12:19:18.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.293+0000 7fc5e2ffd640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5d00678c0 0x7fc5d0069d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4007fd0 tx=0x7fc5d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:18.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.297+0000 7fc5cb7fe640 1 -- 172.21.15.52:0/3206780530 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc5d8016110 con 0x7fc5e40c7f80 2024-05-04T12:19:18.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.517+0000 7fc5e963c640 1 -- 172.21.15.52:0/3206780530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fc5e4075320 con 0x7fc5e40c7f80 2024-05-04T12:19:18.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.521+0000 7fc5cb7fe640 1 -- 172.21.15.52:0/3206780530 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7fc5d80f1270 con 0x7fc5e40c7f80 2024-05-04T12:19:18.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.525+0000 7fc5e963c640 1 -- 172.21.15.52:0/3206780530 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5d00678c0 msgr2=0x7fc5d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.525+0000 7fc5e963c640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5d00678c0 0x7fc5d0069d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4007fd0 tx=0x7fc5d4009040 comp rx=0 tx=0).stop 2024-05-04T12:19:18.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.525+0000 7fc5e963c640 1 -- 172.21.15.52:0/3206780530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e40c7f80 msgr2=0x7fc5e41fa230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.525+0000 7fc5e963c640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e40c7f80 0x7fc5e41fa230 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8019070 tx=0x7fc5d801aca0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.525+0000 7fc5e963c640 1 -- 172.21.15.52:0/3206780530 shutdown_connections 2024-05-04T12:19:18.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.525+0000 7fc5e963c640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5d00678c0 0x7fc5d0069d80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.525+0000 7fc5e963c640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5e41f8620 0x7fc5e41f8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.525+0000 7fc5e963c640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5e40c7f80 0x7fc5e41fa230 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.525+0000 7fc5e963c640 1 --2- 172.21.15.52:0/3206780530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e40706d0 0x7fc5e4139720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.525+0000 7fc5e963c640 1 -- 172.21.15.52:0/3206780530 >> 172.21.15.52:0/3206780530 conn(0x7fc5e4067070 msgr2=0x7fc5e40d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:18.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.525+0000 7fc5e963c640 1 -- 172.21.15.52:0/3206780530 shutdown_connections 2024-05-04T12:19:18.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.525+0000 7fc5e963c640 1 -- 172.21.15.52:0/3206780530 wait complete. 2024-05-04T12:19:18.544 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-04T12:19:18.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55bb92e640 1 Processor -- start 2024-05-04T12:19:18.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55bb92e640 1 -- start start 2024-05-04T12:19:18.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55bb92e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55b41441b0 0x7f55b41627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:18.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55bb92e640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55b4162cf0 con 0x7f55b40b6bf0 2024-05-04T12:19:18.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55bb92e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55b415eac0 con 0x7f55b4144590 2024-05-04T12:19:18.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55bb92e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f55b415f6d0 con 0x7f55b41441b0 2024-05-04T12:19:18.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55b9ecb640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f55b40b6bf0 0x7f55b4141ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:37740/0 (socket says 172.21.15.52:37740) 2024-05-04T12:19:18.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55b9ecb640 1 -- 172.21.15.52:0/3435256575 learned_addr learned my addr 172.21.15.52:0/3435256575 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:18.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 -- 172.21.15.52:0/3435256575 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1565787435 0 0) 0x7f55ac0014a0 con 0x7f55b40b6bf0 2024-05-04T12:19:18.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55b96ca640 1 --2- 172.21.15.52:0/3435256575 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55b41441b0 0x7f55b41627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:18.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 -- 172.21.15.52:0/3435256575 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55a0003530 con 0x7f55b40b6bf0 2024-05-04T12:19:18.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 -- 172.21.15.52:0/3435256575 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1490571264 0 0) 0x7f55a0003530 con 0x7f55b40b6bf0 2024-05-04T12:19:18.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 -- 172.21.15.52:0/3435256575 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55b41441b0 msgr2=0x7f55b41627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:18.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 --2- 172.21.15.52:0/3435256575 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55b41441b0 0x7f55b41627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 -- 172.21.15.52:0/3435256575 >> v1:172.21.15.111:6789/0 conn(0x7f55b4144590 legacy=0x7f55b40b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 -- 172.21.15.52:0/3435256575 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b415eda0 con 0x7f55b40b6bf0 2024-05-04T12:19:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 -- 172.21.15.52:0/3435256575 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f55ac001bd0 con 0x7f55b40b6bf0 2024-05-04T12:19:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 -- 172.21.15.52:0/3435256575 >> v1:172.21.15.52:6789/0 conn(0x7f55b40b6bf0 legacy=0x7f55b4141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 --2- 172.21.15.52:0/3435256575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55a00039f0 0x7f55a0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 --2- 172.21.15.52:0/3435256575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55a0004810 0x7f55a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 --2- 172.21.15.52:0/3435256575 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55b41441b0 0x7f55a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 -- 172.21.15.52:0/3435256575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f55b415f6d0 con 0x7f55a00039f0 2024-05-04T12:19:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 -- 172.21.15.52:0/3435256575 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f55a000ba80 con 0x7f55a0004810 2024-05-04T12:19:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55abfff640 1 -- 172.21.15.52:0/3435256575 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f55a000bc80 con 0x7f55b41441b0 2024-05-04T12:19:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.657+0000 7f55b96ca640 1 --2- 172.21.15.52:0/3435256575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55a00039f0 0x7f55a0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55b8ec9640 1 --2- 172.21.15.52:0/3435256575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55a0004810 0x7f55a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55b9ecb640 1 --2- 172.21.15.52:0/3435256575 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55b41441b0 0x7f55a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55b96ca640 1 -- 172.21.15.52:0/3435256575 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55b41441b0 msgr2=0x7f55a00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55b96ca640 1 --2- 172.21.15.52:0/3435256575 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55b41441b0 0x7f55a00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55b96ca640 1 -- 172.21.15.52:0/3435256575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55a0004810 msgr2=0x7f55a0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:18.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55b96ca640 1 --2- 172.21.15.52:0/3435256575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55a0004810 0x7f55a0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55b96ca640 1 -- 172.21.15.52:0/3435256575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a0003530 con 0x7f55a00039f0 2024-05-04T12:19:18.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55b96ca640 1 --2- 172.21.15.52:0/3435256575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55a00039f0 0x7f55a0003e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009ab0 tx=0x7f55b001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6f786736d8e703 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:18.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55abfff640 1 -- 172.21.15.52:0/3435256575 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b00096b0 con 0x7f55a00039f0 2024-05-04T12:19:18.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55abfff640 1 -- 172.21.15.52:0/3435256575 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b00a64f0 con 0x7f55a00039f0 2024-05-04T12:19:18.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55abfff640 1 -- 172.21.15.52:0/3435256575 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b00a68e0 con 0x7f55a00039f0 2024-05-04T12:19:18.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55bb92e640 1 -- 172.21.15.52:0/3435256575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55a00039f0 msgr2=0x7f55a0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55bb92e640 1 --2- 172.21.15.52:0/3435256575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55a00039f0 0x7f55a0003e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009ab0 tx=0x7f55b001ca00 comp rx=0 tx=0).stop 2024-05-04T12:19:18.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55b9ecb640 1 -- 172.21.15.52:0/3435256575 reap_dead start 2024-05-04T12:19:18.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55bb92e640 1 -- 172.21.15.52:0/3435256575 shutdown_connections 2024-05-04T12:19:18.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55bb92e640 1 -- 172.21.15.52:0/3435256575 >> 172.21.15.52:0/3435256575 conn(0x7f55b40b52d0 msgr2=0x7f55b40b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:18.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55bb92e640 1 -- 172.21.15.52:0/3435256575 shutdown_connections 2024-05-04T12:19:18.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.661+0000 7f55bb92e640 1 -- 172.21.15.52:0/3435256575 wait complete. 2024-05-04T12:19:18.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55bb92e640 1 Processor -- start 2024-05-04T12:19:18.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55bb92e640 1 -- start start 2024-05-04T12:19:18.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55bb92e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55a00039f0 0x7f55b41548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:18.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55bb92e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55b4131310 0x7f55b4131790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:18.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55bb92e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b41ea9c0 0x7f55b41eae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:18.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55bb92e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f55b4143c10 con 0x7f55a00039f0 2024-05-04T12:19:18.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55bb92e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f55b4143a90 con 0x7f55b41ea9c0 2024-05-04T12:19:18.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55bb92e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f55b41429d0 con 0x7f55b4131310 2024-05-04T12:19:18.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55b96ca640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55a00039f0 0x7f55b41548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:18.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55b96ca640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55a00039f0 0x7f55b41548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43688/0 (socket says 172.21.15.52:43688) 2024-05-04T12:19:18.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55b96ca640 1 -- 172.21.15.52:0/5588623 learned_addr learned my addr 172.21.15.52:0/5588623 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:18.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55b8ec9640 1 --2- 172.21.15.52:0/5588623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55b4131310 0x7f55b4131790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:18.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55b9ecb640 1 --2- 172.21.15.52:0/5588623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b41ea9c0 0x7f55b41eae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:18.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55b96ca640 1 -- 172.21.15.52:0/5588623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55b4131310 msgr2=0x7f55b4131790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55b96ca640 1 --2- 172.21.15.52:0/5588623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55b4131310 0x7f55b4131790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55b96ca640 1 -- 172.21.15.52:0/5588623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b41ea9c0 msgr2=0x7f55b41eae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55b96ca640 1 --2- 172.21.15.52:0/5588623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b41ea9c0 0x7f55b41eae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55b96ca640 1 -- 172.21.15.52:0/5588623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b41eb760 con 0x7f55a00039f0 2024-05-04T12:19:18.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55b96ca640 1 --2- 172.21.15.52:0/5588623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55a00039f0 0x7f55b41548d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f55b00b0040 tx=0x7f55a0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:18.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55a9ffb640 1 -- 172.21.15.52:0/5588623 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b00a66a0 con 0x7f55a00039f0 2024-05-04T12:19:18.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55a9ffb640 1 -- 172.21.15.52:0/5588623 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b00bf070 con 0x7f55a00039f0 2024-05-04T12:19:18.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55bb92e640 1 -- 172.21.15.52:0/5588623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b415bb30 con 0x7f55a00039f0 2024-05-04T12:19:18.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55a9ffb640 1 -- 172.21.15.52:0/5588623 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b00a6aa0 con 0x7f55a00039f0 2024-05-04T12:19:18.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55bb92e640 1 -- 172.21.15.52:0/5588623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b415c0f0 con 0x7f55a00039f0 2024-05-04T12:19:18.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.665+0000 7f55bb92e640 1 -- 172.21.15.52:0/5588623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b4062e50 con 0x7f55a00039f0 2024-05-04T12:19:18.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.669+0000 7f55a9ffb640 1 -- 172.21.15.52:0/5588623 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f55b00a6c40 con 0x7f55a00039f0 2024-05-04T12:19:18.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.669+0000 7f55a9ffb640 1 --2- 172.21.15.52:0/5588623 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f55980678f0 0x7f5598069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:18.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.669+0000 7f55a9ffb640 1 -- 172.21.15.52:0/5588623 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f55b0120200 con 0x7f55a00039f0 2024-05-04T12:19:18.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.669+0000 7f55b8ec9640 1 --2- 172.21.15.52:0/5588623 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f55980678f0 0x7f5598069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:18.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.669+0000 7f55b8ec9640 1 --2- 172.21.15.52:0/5588623 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f55980678f0 0x7f5598069db0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f55a4000c10 tx=0x7f55a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:18.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.673+0000 7f55a9ffb640 1 -- 172.21.15.52:0/5588623 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f55b0016740 con 0x7f55a00039f0 2024-05-04T12:19:18.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.893+0000 7f55bb92e640 1 -- 172.21.15.52:0/5588623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f55b4141550 con 0x7f55a00039f0 2024-05-04T12:19:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.897+0000 7f55a9ffb640 1 -- 172.21.15.52:0/5588623 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) ==== 97+0+0 (secure 0 0 0) 0x7f55b00f1090 con 0x7f55a00039f0 2024-05-04T12:19:18.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.901+0000 7f55bb92e640 1 -- 172.21.15.52:0/5588623 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f55980678f0 msgr2=0x7f5598069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.901+0000 7f55bb92e640 1 --2- 172.21.15.52:0/5588623 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f55980678f0 0x7f5598069db0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f55a4000c10 tx=0x7f55a4009040 comp rx=0 tx=0).stop 2024-05-04T12:19:18.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.901+0000 7f55bb92e640 1 -- 172.21.15.52:0/5588623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55a00039f0 msgr2=0x7f55b41548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:18.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.901+0000 7f55bb92e640 1 --2- 172.21.15.52:0/5588623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55a00039f0 0x7f55b41548d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f55b00b0040 tx=0x7f55a0005840 comp rx=0 tx=0).stop 2024-05-04T12:19:18.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.901+0000 7f55bb92e640 1 -- 172.21.15.52:0/5588623 shutdown_connections 2024-05-04T12:19:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.901+0000 7f55bb92e640 1 --2- 172.21.15.52:0/5588623 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f55980678f0 0x7f5598069db0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.901+0000 7f55bb92e640 1 --2- 172.21.15.52:0/5588623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b41ea9c0 0x7f55b41eae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.901+0000 7f55bb92e640 1 --2- 172.21.15.52:0/5588623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55b4131310 0x7f55b4131790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.901+0000 7f55bb92e640 1 --2- 172.21.15.52:0/5588623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55a00039f0 0x7f55b41548d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.901+0000 7f55bb92e640 1 -- 172.21.15.52:0/5588623 >> 172.21.15.52:0/5588623 conn(0x7f55b40b52d0 msgr2=0x7f55b40d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.901+0000 7f55bb92e640 1 -- 172.21.15.52:0/5588623 shutdown_connections 2024-05-04T12:19:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:18.901+0000 7f55bb92e640 1 -- 172.21.15.52:0/5588623 wait complete. 2024-05-04T12:19:18.920 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-04T12:19:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd1833a640 1 Processor -- start 2024-05-04T12:19:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd1833a640 1 -- start start 2024-05-04T12:19:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd1833a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efd100ce160 0x7efd100ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd1833a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7efd100d02f0 0x7efd100d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd1833a640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd10147d40 con 0x7efd10147960 2024-05-04T12:19:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd1833a640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7efd10073830 con 0x7efd100ce160 2024-05-04T12:19:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd1833a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7efd10072790 con 0x7efd100d02f0 2024-05-04T12:19:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd158d5640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efd100ce160 0x7efd100ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd158d5640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efd100ce160 0x7efd100ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43700/0 (socket says 172.21.15.52:43700) 2024-05-04T12:19:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd158d5640 1 -- 172.21.15.52:0/2852966791 learned_addr learned my addr 172.21.15.52:0/2852966791 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd168d7640 1 --2- 172.21.15.52:0/2852966791 >> v2:172.21.15.111:3300/0 conn(0x7efd100d02f0 0x7efd100d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd158d5640 1 -- 172.21.15.52:0/2852966791 >> v2:172.21.15.111:3300/0 conn(0x7efd100d02f0 msgr2=0x7efd100d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd158d5640 1 --2- 172.21.15.52:0/2852966791 >> v2:172.21.15.111:3300/0 conn(0x7efd100d02f0 0x7efd100d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd158d5640 1 -- 172.21.15.52:0/2852966791 >> v1:172.21.15.52:6789/0 conn(0x7efd10147960 legacy=0x7efd100cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.037+0000 7efd158d5640 1 -- 172.21.15.52:0/2852966791 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd10072a70 con 0x7efd100ce160 2024-05-04T12:19:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.041+0000 7efd158d5640 1 --2- 172.21.15.52:0/2852966791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd100ce160 0x7efd100ccdf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efd101632a0 tx=0x7efd0001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d904c0e02a2dead server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.041+0000 7efd150d4640 1 -- 172.21.15.52:0/2852966791 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd000ab610 con 0x7efd100ce160 2024-05-04T12:19:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.041+0000 7efd150d4640 1 -- 172.21.15.52:0/2852966791 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd000b0070 con 0x7efd100ce160 2024-05-04T12:19:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.041+0000 7efd150d4640 1 -- 172.21.15.52:0/2852966791 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd000ab9d0 con 0x7efd100ce160 2024-05-04T12:19:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.041+0000 7efd1833a640 1 -- 172.21.15.52:0/2852966791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd100ce160 msgr2=0x7efd100ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.041+0000 7efd1833a640 1 --2- 172.21.15.52:0/2852966791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd100ce160 0x7efd100ccdf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efd101632a0 tx=0x7efd0001c520 comp rx=0 tx=0).stop 2024-05-04T12:19:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.041+0000 7efd1833a640 1 -- 172.21.15.52:0/2852966791 shutdown_connections 2024-05-04T12:19:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.041+0000 7efd1833a640 1 --2- 172.21.15.52:0/2852966791 >> v2:172.21.15.111:3300/0 conn(0x7efd100d02f0 0x7efd100d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.041+0000 7efd1833a640 1 --2- 172.21.15.52:0/2852966791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd100ce160 0x7efd100ccdf0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.041+0000 7efd1833a640 1 -- 172.21.15.52:0/2852966791 >> 172.21.15.52:0/2852966791 conn(0x7efd10070690 msgr2=0x7efd10070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.041+0000 7efd1833a640 1 -- 172.21.15.52:0/2852966791 shutdown_connections 2024-05-04T12:19:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.041+0000 7efd1833a640 1 -- 172.21.15.52:0/2852966791 wait complete. 2024-05-04T12:19:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.041+0000 7efd1833a640 1 Processor -- start 2024-05-04T12:19:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.041+0000 7efd1833a640 1 -- start start 2024-05-04T12:19:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd1833a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd100ce160 0x7efd101457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd1833a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd100d02f0 0x7efd1015de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd1833a640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd10147960 0x7efd101406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd1833a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efd10148250 con 0x7efd100ce160 2024-05-04T12:19:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd1833a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efd101480d0 con 0x7efd100d02f0 2024-05-04T12:19:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd1833a640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7efd10072320 con 0x7efd10147960 2024-05-04T12:19:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd160d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd100ce160 0x7efd101457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd160d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd100ce160 0x7efd101457a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43710/0 (socket says 172.21.15.52:43710) 2024-05-04T12:19:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd160d6640 1 -- 172.21.15.52:0/1700062076 learned_addr learned my addr 172.21.15.52:0/1700062076 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd158d5640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd100d02f0 0x7efd1015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd168d7640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd10147960 0x7efd101406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd160d6640 1 -- 172.21.15.52:0/1700062076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd10147960 msgr2=0x7efd101406e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd160d6640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd10147960 0x7efd101406e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd160d6640 1 -- 172.21.15.52:0/1700062076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd100d02f0 msgr2=0x7efd1015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd160d6640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd100d02f0 0x7efd1015de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd160d6640 1 -- 172.21.15.52:0/1700062076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd10139ed0 con 0x7efd100ce160 2024-05-04T12:19:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd158d5640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd100d02f0 0x7efd1015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd160d6640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd100ce160 0x7efd101457a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00c3c0 tx=0x7efd0c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd06ffd640 1 -- 172.21.15.52:0/1700062076 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c017070 con 0x7efd100ce160 2024-05-04T12:19:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd06ffd640 1 -- 172.21.15.52:0/1700062076 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd0c0052f0 con 0x7efd100ce160 2024-05-04T12:19:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd06ffd640 1 -- 172.21.15.52:0/1700062076 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c003840 con 0x7efd100ce160 2024-05-04T12:19:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd1833a640 1 -- 172.21.15.52:0/1700062076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1013a1b0 con 0x7efd100ce160 2024-05-04T12:19:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd1833a640 1 -- 172.21.15.52:0/1700062076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1013a770 con 0x7efd100ce160 2024-05-04T12:19:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd1833a640 1 -- 172.21.15.52:0/1700062076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd10062e50 con 0x7efd100ce160 2024-05-04T12:19:19.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd06ffd640 1 -- 172.21.15.52:0/1700062076 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efd0c01a070 con 0x7efd100ce160 2024-05-04T12:19:19.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd06ffd640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7efcf40678c0 0x7efcf4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd06ffd640 1 -- 172.21.15.52:0/1700062076 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7efd0c084c10 con 0x7efd100ce160 2024-05-04T12:19:19.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.045+0000 7efd158d5640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7efcf40678c0 0x7efcf4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.049+0000 7efd158d5640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7efcf40678c0 0x7efcf4069d80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efd000af000 tx=0x7efd000a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:19.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.049+0000 7efd06ffd640 1 -- 172.21.15.52:0/1700062076 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efd10147d40 con 0x7efd100ce160 2024-05-04T12:19:19.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.269+0000 7efd1833a640 1 -- 172.21.15.52:0/1700062076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7efd10075320 con 0x7efd100ce160 2024-05-04T12:19:19.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.273+0000 7efd06ffd640 1 -- 172.21.15.52:0/1700062076 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) ==== 90+0+0 (secure 0 0 0) 0x7efd0c055b10 con 0x7efd100ce160 2024-05-04T12:19:19.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.277+0000 7efd1833a640 1 -- 172.21.15.52:0/1700062076 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7efcf40678c0 msgr2=0x7efcf4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.277+0000 7efd1833a640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7efcf40678c0 0x7efcf4069d80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efd000af000 tx=0x7efd000a7000 comp rx=0 tx=0).stop 2024-05-04T12:19:19.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.277+0000 7efd1833a640 1 -- 172.21.15.52:0/1700062076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd100ce160 msgr2=0x7efd101457a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.277+0000 7efd1833a640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd100ce160 0x7efd101457a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00c3c0 tx=0x7efd0c00c880 comp rx=0 tx=0).stop 2024-05-04T12:19:19.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.277+0000 7efd1833a640 1 -- 172.21.15.52:0/1700062076 shutdown_connections 2024-05-04T12:19:19.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.277+0000 7efd1833a640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7efcf40678c0 0x7efcf4069d80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.277+0000 7efd1833a640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd10147960 0x7efd101406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.277+0000 7efd1833a640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd100d02f0 0x7efd1015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.277+0000 7efd1833a640 1 --2- 172.21.15.52:0/1700062076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd100ce160 0x7efd101457a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.277+0000 7efd1833a640 1 -- 172.21.15.52:0/1700062076 >> 172.21.15.52:0/1700062076 conn(0x7efd10070690 msgr2=0x7efd100ae660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:19.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.277+0000 7efd1833a640 1 -- 172.21.15.52:0/1700062076 shutdown_connections 2024-05-04T12:19:19.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.277+0000 7efd1833a640 1 -- 172.21.15.52:0/1700062076 wait complete. 2024-05-04T12:19:19.299 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:19:19.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.413+0000 7f1c14055640 1 Processor -- start 2024-05-04T12:19:19.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.413+0000 7f1c14055640 1 -- start start 2024-05-04T12:19:19.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.413+0000 7f1c14055640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1c0c13f000 0x7f1c0c06b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.413+0000 7f1c14055640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c0c13f3e0 con 0x7f1c0c160260 2024-05-04T12:19:19.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.413+0000 7f1c14055640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c0c15c560 con 0x7f1c0c06b750 2024-05-04T12:19:19.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.413+0000 7f1c14055640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f1c0c160670 con 0x7f1c0c13f000 2024-05-04T12:19:19.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c11df1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1c0c13f000 0x7f1c0c06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c125f2640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f1c0c160260 0x7f1c0c0a9480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:37756/0 (socket says 172.21.15.52:37756) 2024-05-04T12:19:19.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c11df1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1c0c13f000 0x7f1c0c06b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43716/0 (socket says 172.21.15.52:43716) 2024-05-04T12:19:19.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c125f2640 1 -- 172.21.15.52:0/3026431114 learned_addr learned my addr 172.21.15.52:0/3026431114 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:19.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c11df1640 1 -- 172.21.15.52:0/3026431114 >> v1:172.21.15.111:6789/0 conn(0x7f1c0c06b750 legacy=0x7f1c0c15fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:19.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c11df1640 1 -- 172.21.15.52:0/3026431114 >> v1:172.21.15.52:6789/0 conn(0x7f1c0c160260 legacy=0x7f1c0c0a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c11df1640 1 -- 172.21.15.52:0/3026431114 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0c1608c0 con 0x7f1c0c13f000 2024-05-04T12:19:19.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c11df1640 1 --2- 172.21.15.52:0/3026431114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c0c13f000 0x7f1c0c06b210 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1c080097c0 tx=0x7f1c0801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6264be78005303 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:19.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c10def640 1 -- 172.21.15.52:0/3026431114 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c080b0070 con 0x7f1c0c13f000 2024-05-04T12:19:19.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c10def640 1 -- 172.21.15.52:0/3026431114 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c080ab600 con 0x7f1c0c13f000 2024-05-04T12:19:19.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c10def640 1 -- 172.21.15.52:0/3026431114 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c080ab9d0 con 0x7f1c0c13f000 2024-05-04T12:19:19.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c14055640 1 -- 172.21.15.52:0/3026431114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c0c13f000 msgr2=0x7f1c0c06b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c14055640 1 --2- 172.21.15.52:0/3026431114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c0c13f000 0x7f1c0c06b210 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1c080097c0 tx=0x7f1c0801c260 comp rx=0 tx=0).stop 2024-05-04T12:19:19.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c14055640 1 -- 172.21.15.52:0/3026431114 shutdown_connections 2024-05-04T12:19:19.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c14055640 1 --2- 172.21.15.52:0/3026431114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c0c13f000 0x7f1c0c06b210 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c14055640 1 -- 172.21.15.52:0/3026431114 >> 172.21.15.52:0/3026431114 conn(0x7f1c0c070690 msgr2=0x7f1c0c070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:19.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c14055640 1 -- 172.21.15.52:0/3026431114 shutdown_connections 2024-05-04T12:19:19.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.417+0000 7f1c14055640 1 -- 172.21.15.52:0/3026431114 wait complete. 2024-05-04T12:19:19.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c14055640 1 Processor -- start 2024-05-04T12:19:19.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c14055640 1 -- start start 2024-05-04T12:19:19.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c14055640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1c0c06b750 0x7f1c0c141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c14055640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c0c13f000 0x7f1c0c143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c14055640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c0c160260 0x7f1c0c1e66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c14055640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c15cde0 con 0x7f1c0c13f000 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c14055640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c15cb00 con 0x7f1c0c160260 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c14055640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f1c0c15d300 con 0x7f1c0c06b750 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c115f0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c0c13f000 0x7f1c0c143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c115f0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c0c13f000 0x7f1c0c143220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43720/0 (socket says 172.21.15.52:43720) 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c115f0640 1 -- 172.21.15.52:0/1944241891 learned_addr learned my addr 172.21.15.52:0/1944241891 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c11df1640 1 --2- 172.21.15.52:0/1944241891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1c0c06b750 0x7f1c0c141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c125f2640 1 --2- 172.21.15.52:0/1944241891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c0c160260 0x7f1c0c1e66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c115f0640 1 -- 172.21.15.52:0/1944241891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1c0c06b750 msgr2=0x7f1c0c141ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c115f0640 1 --2- 172.21.15.52:0/1944241891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1c0c06b750 0x7f1c0c141ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c115f0640 1 -- 172.21.15.52:0/1944241891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c0c160260 msgr2=0x7f1c0c1e66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c115f0640 1 --2- 172.21.15.52:0/1944241891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c0c160260 0x7f1c0c1e66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c115f0640 1 -- 172.21.15.52:0/1944241891 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0c1e5940 con 0x7f1c0c13f000 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c115f0640 1 --2- 172.21.15.52:0/1944241891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c0c13f000 0x7f1c0c143220 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b440 tx=0x7f1bfc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:19.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c027fc640 1 -- 172.21.15.52:0/1944241891 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc015020 con 0x7f1c0c13f000 2024-05-04T12:19:19.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c027fc640 1 -- 172.21.15.52:0/1944241891 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bfc011890 con 0x7f1c0c13f000 2024-05-04T12:19:19.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c14055640 1 -- 172.21.15.52:0/1944241891 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0c143960 con 0x7f1c0c13f000 2024-05-04T12:19:19.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c027fc640 1 -- 172.21.15.52:0/1944241891 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc011c80 con 0x7f1c0c13f000 2024-05-04T12:19:19.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.421+0000 7f1c14055640 1 -- 172.21.15.52:0/1944241891 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0c1e5ee0 con 0x7f1c0c13f000 2024-05-04T12:19:19.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.425+0000 7f1c14055640 1 -- 172.21.15.52:0/1944241891 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c0c062e50 con 0x7f1c0c13f000 2024-05-04T12:19:19.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.425+0000 7f1c027fc640 1 -- 172.21.15.52:0/1944241891 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1bfc018020 con 0x7f1c0c13f000 2024-05-04T12:19:19.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.425+0000 7f1c027fc640 1 --2- 172.21.15.52:0/1944241891 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1bf00678c0 0x7f1bf0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.425+0000 7f1c11df1640 1 --2- 172.21.15.52:0/1944241891 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1bf00678c0 0x7f1bf0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.425+0000 7f1c027fc640 1 -- 172.21.15.52:0/1944241891 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f1bfc085730 con 0x7f1c0c13f000 2024-05-04T12:19:19.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.425+0000 7f1c11df1640 1 --2- 172.21.15.52:0/1944241891 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1bf00678c0 0x7f1bf0069d80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1c080ad000 tx=0x7f1c080a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:19.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.429+0000 7f1c027fc640 1 -- 172.21.15.52:0/1944241891 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1bfc01f030 con 0x7f1c0c13f000 2024-05-04T12:19:19.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.585+0000 7f1c14055640 1 -- 172.21.15.52:0/1944241891 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c0c075320 con 0x7f1c0c13f000 2024-05-04T12:19:19.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.589+0000 7f1c027fc640 1 -- 172.21.15.52:0/1944241891 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+12903 (secure 0 0 0) 0x7f1bfc00d040 con 0x7f1c0c13f000 2024-05-04T12:19:19.592 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:19.592 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:19:16.616837+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:19:19.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.589+0000 7f1c14055640 1 -- 172.21.15.52:0/1944241891 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1bf00678c0 msgr2=0x7f1bf0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.589+0000 7f1c14055640 1 --2- 172.21.15.52:0/1944241891 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1bf00678c0 0x7f1bf0069d80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1c080ad000 tx=0x7f1c080a6000 comp rx=0 tx=0).stop 2024-05-04T12:19:19.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.589+0000 7f1c14055640 1 -- 172.21.15.52:0/1944241891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c0c13f000 msgr2=0x7f1c0c143220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.589+0000 7f1c14055640 1 --2- 172.21.15.52:0/1944241891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c0c13f000 0x7f1c0c143220 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b440 tx=0x7f1bfc00b900 comp rx=0 tx=0).stop 2024-05-04T12:19:19.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.589+0000 7f1c14055640 1 -- 172.21.15.52:0/1944241891 shutdown_connections 2024-05-04T12:19:19.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.589+0000 7f1c14055640 1 --2- 172.21.15.52:0/1944241891 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f1bf00678c0 0x7f1bf0069d80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.589+0000 7f1c14055640 1 --2- 172.21.15.52:0/1944241891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c0c160260 0x7f1c0c1e66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.589+0000 7f1c14055640 1 --2- 172.21.15.52:0/1944241891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c0c13f000 0x7f1c0c143220 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.589+0000 7f1c14055640 1 --2- 172.21.15.52:0/1944241891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1c0c06b750 0x7f1c0c141ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.589+0000 7f1c14055640 1 -- 172.21.15.52:0/1944241891 >> 172.21.15.52:0/1944241891 conn(0x7f1c0c070690 msgr2=0x7f1c0c0d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:19.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.589+0000 7f1c14055640 1 -- 172.21.15.52:0/1944241891 shutdown_connections 2024-05-04T12:19:19.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.589+0000 7f1c14055640 1 -- 172.21.15.52:0/1944241891 wait complete. 2024-05-04T12:19:19.610 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T12:19:19.610 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T12:19:19.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda081c5640 1 Processor -- start 2024-05-04T12:19:19.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda081c5640 1 -- start start 2024-05-04T12:19:19.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda081c5640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda00144570 0x7fda00143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda081c5640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fda00143e80 0x7fda001418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda081c5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda000d2710 con 0x7fda00141010 2024-05-04T12:19:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda081c5640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fda000d29c0 con 0x7fda00143e80 2024-05-04T12:19:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda081c5640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fda000d2ca0 con 0x7fda00144570 2024-05-04T12:19:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda06762640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fda00143e80 0x7fda001418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda06762640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fda00143e80 0x7fda001418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43732/0 (socket says 172.21.15.52:43732) 2024-05-04T12:19:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda06762640 1 -- 172.21.15.52:0/891863765 learned_addr learned my addr 172.21.15.52:0/891863765 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda05760640 1 --2- 172.21.15.52:0/891863765 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda00144570 0x7fda00143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda06762640 1 -- 172.21.15.52:0/891863765 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda00144570 msgr2=0x7fda00143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda06762640 1 --2- 172.21.15.52:0/891863765 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda00144570 0x7fda00143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda06762640 1 -- 172.21.15.52:0/891863765 >> v1:172.21.15.111:6789/0 conn(0x7fda00141010 legacy=0x7fda000d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda06762640 1 -- 172.21.15.52:0/891863765 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda000d2f80 con 0x7fda00143e80 2024-05-04T12:19:19.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.721+0000 7fda05760640 1 --2- 172.21.15.52:0/891863765 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda00144570 0x7fda00143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda06762640 1 --2- 172.21.15.52:0/891863765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda00143e80 0x7fda001418b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009e50 tx=0x7fd9f801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32a8004e9cf09f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda04f5f640 1 -- 172.21.15.52:0/891863765 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80a56d0 con 0x7fda00143e80 2024-05-04T12:19:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda04f5f640 1 -- 172.21.15.52:0/891863765 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f80a5870 con 0x7fda00143e80 2024-05-04T12:19:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda04f5f640 1 -- 172.21.15.52:0/891863765 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80a5c40 con 0x7fda00143e80 2024-05-04T12:19:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 -- 172.21.15.52:0/891863765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda00143e80 msgr2=0x7fda001418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 --2- 172.21.15.52:0/891863765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda00143e80 0x7fda001418b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009e50 tx=0x7fd9f801c960 comp rx=0 tx=0).stop 2024-05-04T12:19:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 -- 172.21.15.52:0/891863765 shutdown_connections 2024-05-04T12:19:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 --2- 172.21.15.52:0/891863765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda00143e80 0x7fda001418b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 --2- 172.21.15.52:0/891863765 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda00144570 0x7fda00143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 -- 172.21.15.52:0/891863765 >> 172.21.15.52:0/891863765 conn(0x7fda00072be0 msgr2=0x7fda000d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 -- 172.21.15.52:0/891863765 shutdown_connections 2024-05-04T12:19:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 -- 172.21.15.52:0/891863765 wait complete. 2024-05-04T12:19:19.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 Processor -- start 2024-05-04T12:19:19.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 -- start start 2024-05-04T12:19:19.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda00141010 0x7fda0013bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda00143e80 0x7fda00154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda00144570 0x7fda0013fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fda00142780 con 0x7fda00144570 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fda00142600 con 0x7fda00141010 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda081c5640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fda00142900 con 0x7fda00143e80 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda06762640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda00144570 0x7fda0013fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda06762640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda00144570 0x7fda0013fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43748/0 (socket says 172.21.15.52:43748) 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda06762640 1 -- 172.21.15.52:0/419278696 learned_addr learned my addr 172.21.15.52:0/419278696 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda06762640 1 -- 172.21.15.52:0/419278696 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda00143e80 msgr2=0x7fda00154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda06762640 1 --2- 172.21.15.52:0/419278696 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda00143e80 0x7fda00154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda06762640 1 -- 172.21.15.52:0/419278696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda00141010 msgr2=0x7fda0013bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda06762640 1 --2- 172.21.15.52:0/419278696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda00141010 0x7fda0013bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.725+0000 7fda06762640 1 -- 172.21.15.52:0/419278696 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0013abe0 con 0x7fda00144570 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.729+0000 7fda06762640 1 --2- 172.21.15.52:0/419278696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda00144570 0x7fda0013fb30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009e50 tx=0x7fd9f8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.729+0000 7fd9f67fc640 1 -- 172.21.15.52:0/419278696 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80bc020 con 0x7fda00144570 2024-05-04T12:19:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.729+0000 7fd9f67fc640 1 -- 172.21.15.52:0/419278696 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f80b8670 con 0x7fda00144570 2024-05-04T12:19:19.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.729+0000 7fda081c5640 1 -- 172.21.15.52:0/419278696 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0013aec0 con 0x7fda00144570 2024-05-04T12:19:19.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.729+0000 7fd9f67fc640 1 -- 172.21.15.52:0/419278696 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80b8a60 con 0x7fda00144570 2024-05-04T12:19:19.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.729+0000 7fda081c5640 1 -- 172.21.15.52:0/419278696 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0013b480 con 0x7fda00144570 2024-05-04T12:19:19.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.729+0000 7fda081c5640 1 -- 172.21.15.52:0/419278696 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda00062e50 con 0x7fda00144570 2024-05-04T12:19:19.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.729+0000 7fd9f67fc640 1 -- 172.21.15.52:0/419278696 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd9f80bf020 con 0x7fda00144570 2024-05-04T12:19:19.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.729+0000 7fd9f67fc640 1 --2- 172.21.15.52:0/419278696 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd9e40678c0 0x7fd9e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:19.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.729+0000 7fd9f67fc640 1 -- 172.21.15.52:0/419278696 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fd9f81209f0 con 0x7fda00144570 2024-05-04T12:19:19.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.729+0000 7fda05f61640 1 --2- 172.21.15.52:0/419278696 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd9e40678c0 0x7fd9e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:19.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.729+0000 7fda05f61640 1 --2- 172.21.15.52:0/419278696 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd9e40678c0 0x7fd9e4069d80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda00139060 tx=0x7fd9fc006bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:19.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.733+0000 7fd9f67fc640 1 -- 172.21.15.52:0/419278696 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd9f80c6030 con 0x7fda00144570 2024-05-04T12:19:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.893+0000 7fda081c5640 1 -- 172.21.15.52:0/419278696 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda000d4df0 con 0x7fda00144570 2024-05-04T12:19:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.893+0000 7fd9f67fc640 1 -- 172.21.15.52:0/419278696 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) ==== 93+0+10 (secure 0 0 0) 0x7fd9f80165e0 con 0x7fda00144570 2024-05-04T12:19:19.896 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T12:19:19.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.893+0000 7fda081c5640 1 -- 172.21.15.52:0/419278696 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd9e40678c0 msgr2=0x7fd9e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.893+0000 7fda081c5640 1 --2- 172.21.15.52:0/419278696 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd9e40678c0 0x7fd9e4069d80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda00139060 tx=0x7fd9fc006bd0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.893+0000 7fda081c5640 1 -- 172.21.15.52:0/419278696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda00144570 msgr2=0x7fda0013fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:19.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.893+0000 7fda081c5640 1 --2- 172.21.15.52:0/419278696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda00144570 0x7fda0013fb30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009e50 tx=0x7fd9f8019700 comp rx=0 tx=0).stop 2024-05-04T12:19:19.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.893+0000 7fda081c5640 1 -- 172.21.15.52:0/419278696 shutdown_connections 2024-05-04T12:19:19.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.893+0000 7fda081c5640 1 --2- 172.21.15.52:0/419278696 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd9e40678c0 0x7fd9e4069d80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.893+0000 7fda081c5640 1 --2- 172.21.15.52:0/419278696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda00144570 0x7fda0013fb30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.893+0000 7fda081c5640 1 --2- 172.21.15.52:0/419278696 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda00143e80 0x7fda00154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.893+0000 7fda081c5640 1 --2- 172.21.15.52:0/419278696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda00141010 0x7fda0013bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:19.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.893+0000 7fda081c5640 1 -- 172.21.15.52:0/419278696 >> 172.21.15.52:0/419278696 conn(0x7fda00072be0 msgr2=0x7fda000d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:19.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.893+0000 7fda081c5640 1 -- 172.21.15.52:0/419278696 shutdown_connections 2024-05-04T12:19:19.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:19.893+0000 7fda081c5640 1 -- 172.21.15.52:0/419278696 wait complete. 2024-05-04T12:19:19.914 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-04T12:19:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.033+0000 7fa0d48ce640 1 Processor -- start 2024-05-04T12:19:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.033+0000 7fa0d48ce640 1 -- start start 2024-05-04T12:19:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0d48ce640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa0d015c7f0 0x7fa0d00723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0d48ce640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa0d00cd190 0x7fa0d015c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0d48ce640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0d0072aa0 con 0x7fa0d00ccdb0 2024-05-04T12:19:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0d48ce640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fa0d0070690 con 0x7fa0d00cd190 2024-05-04T12:19:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0d48ce640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d00708e0 con 0x7fa0d015c7f0 2024-05-04T12:19:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0ce59c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa0d00ccdb0 0x7fa0d0158640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:37770/0 (socket says 172.21.15.52:37770) 2024-05-04T12:19:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0ced9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa0d00cd190 0x7fa0d015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0ce59c640 1 -- 172.21.15.52:0/1868277857 learned_addr learned my addr 172.21.15.52:0/1868277857 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0cdd9b640 1 --2- 172.21.15.52:0/1868277857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa0d015c7f0 0x7fa0d00723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0ced9d640 1 -- 172.21.15.52:0/1868277857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa0d015c7f0 msgr2=0x7fa0d00723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0ced9d640 1 --2- 172.21.15.52:0/1868277857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa0d015c7f0 0x7fa0d00723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0ced9d640 1 -- 172.21.15.52:0/1868277857 >> v1:172.21.15.52:6789/0 conn(0x7fa0d00ccdb0 legacy=0x7fa0d0158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0cd59a640 1 -- 172.21.15.52:0/1868277857 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3179247326 0 0) 0x7fa0d0072aa0 con 0x7fa0d00ccdb0 2024-05-04T12:19:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0ced9d640 1 -- 172.21.15.52:0/1868277857 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d0070bc0 con 0x7fa0d00cd190 2024-05-04T12:19:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0ced9d640 1 --2- 172.21.15.52:0/1868277857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d00cd190 0x7fa0d015c120 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009d90 tx=0x7fa0c801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2143b709443d2905 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.037+0000 7fa0cd59a640 1 -- 172.21.15.52:0/1868277857 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c80a8620 con 0x7fa0d00cd190 2024-05-04T12:19:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0cd59a640 1 -- 172.21.15.52:0/1868277857 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c80a87c0 con 0x7fa0d00cd190 2024-05-04T12:19:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0cd59a640 1 -- 172.21.15.52:0/1868277857 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c80a8b90 con 0x7fa0d00cd190 2024-05-04T12:19:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1868277857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d00cd190 msgr2=0x7fa0d015c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 --2- 172.21.15.52:0/1868277857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d00cd190 0x7fa0d015c120 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009d90 tx=0x7fa0c801c8e0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1868277857 shutdown_connections 2024-05-04T12:19:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 --2- 172.21.15.52:0/1868277857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d00cd190 0x7fa0d015c120 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 --2- 172.21.15.52:0/1868277857 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa0d015c7f0 0x7fa0d00723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1868277857 >> 172.21.15.52:0/1868277857 conn(0x7fa0d00b14a0 msgr2=0x7fa0d00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1868277857 shutdown_connections 2024-05-04T12:19:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1868277857 wait complete. 2024-05-04T12:19:20.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 Processor -- start 2024-05-04T12:19:20.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 -- start start 2024-05-04T12:19:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa0d00ccdb0 0x7fa0d014de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d00cd190 0x7fa0d00aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d015c7f0 0x7fa0d00ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d0168a10 con 0x7fa0d00cd190 2024-05-04T12:19:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d0168890 con 0x7fa0d015c7f0 2024-05-04T12:19:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d0168b90 con 0x7fa0d00ccdb0 2024-05-04T12:19:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0cdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d00cd190 0x7fa0d00aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0cdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d00cd190 0x7fa0d00aac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43772/0 (socket says 172.21.15.52:43772) 2024-05-04T12:19:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa0d00ccdb0 0x7fa0d014de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0cdd9b640 1 -- 172.21.15.52:0/1716041210 learned_addr learned my addr 172.21.15.52:0/1716041210 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0ced9d640 1 --2- 172.21.15.52:0/1716041210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d015c7f0 0x7fa0d00ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0cdd9b640 1 -- 172.21.15.52:0/1716041210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa0d00ccdb0 msgr2=0x7fa0d014de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0cdd9b640 1 --2- 172.21.15.52:0/1716041210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa0d00ccdb0 0x7fa0d014de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0cdd9b640 1 -- 172.21.15.52:0/1716041210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d015c7f0 msgr2=0x7fa0d00ab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0cdd9b640 1 --2- 172.21.15.52:0/1716041210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d015c7f0 0x7fa0d00ab1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0cdd9b640 1 -- 172.21.15.52:0/1716041210 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d0159f90 con 0x7fa0d00cd190 2024-05-04T12:19:20.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0cdd9b640 1 --2- 172.21.15.52:0/1716041210 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d00cd190 0x7fa0d00aac80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800d800 tx=0x7fa0b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:20.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0c6ffd640 1 -- 172.21.15.52:0/1716041210 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800b6c0 con 0x7fa0d00cd190 2024-05-04T12:19:20.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0c6ffd640 1 -- 172.21.15.52:0/1716041210 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0b8019070 con 0x7fa0d00cd190 2024-05-04T12:19:20.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1716041210 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d015a270 con 0x7fa0d00cd190 2024-05-04T12:19:20.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0c6ffd640 1 -- 172.21.15.52:0/1716041210 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800ba60 con 0x7fa0d00cd190 2024-05-04T12:19:20.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.041+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1716041210 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d015a830 con 0x7fa0d00cd190 2024-05-04T12:19:20.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.045+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1716041210 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d0062e50 con 0x7fa0d00cd190 2024-05-04T12:19:20.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.045+0000 7fa0c6ffd640 1 -- 172.21.15.52:0/1716041210 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa0b800bc00 con 0x7fa0d00cd190 2024-05-04T12:19:20.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.045+0000 7fa0c6ffd640 1 --2- 172.21.15.52:0/1716041210 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa0b00678c0 0x7fa0b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.045+0000 7fa0c6ffd640 1 -- 172.21.15.52:0/1716041210 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fa0b8085c60 con 0x7fa0d00cd190 2024-05-04T12:19:20.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.045+0000 7fa0ce59c640 1 --2- 172.21.15.52:0/1716041210 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa0b00678c0 0x7fa0b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.045+0000 7fa0ce59c640 1 --2- 172.21.15.52:0/1716041210 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa0b00678c0 0x7fa0b0069d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0000f70 tx=0x7fa0c0007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:20.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.049+0000 7fa0c6ffd640 1 -- 172.21.15.52:0/1716041210 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa0b8056530 con 0x7fa0d00cd190 2024-05-04T12:19:20.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.213+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1716041210 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa0d0075320 con 0x7fa0d00cd190 2024-05-04T12:19:20.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.213+0000 7fa0c6ffd640 1 -- 172.21.15.52:0/1716041210 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) ==== 84+0+65 (secure 0 0 0) 0x7fa0b805a830 con 0x7fa0d00cd190 2024-05-04T12:19:20.218 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:20.219 INFO:teuthology.orchestra.run.smithi052.stdout:[ 2024-05-04T12:19:20.219 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-04T12:19:20.219 INFO:teuthology.orchestra.run.smithi052.stdout: "poolnum": 1, 2024-05-04T12:19:20.219 INFO:teuthology.orchestra.run.smithi052.stdout: "poolname": ".mgr" 2024-05-04T12:19:20.219 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:20.219 INFO:teuthology.orchestra.run.smithi052.stdout:] 2024-05-04T12:19:20.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.217+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1716041210 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa0b00678c0 msgr2=0x7fa0b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.217+0000 7fa0d48ce640 1 --2- 172.21.15.52:0/1716041210 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa0b00678c0 0x7fa0b0069d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0000f70 tx=0x7fa0c0007df0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.217+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1716041210 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d00cd190 msgr2=0x7fa0d00aac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.217+0000 7fa0d48ce640 1 --2- 172.21.15.52:0/1716041210 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d00cd190 0x7fa0d00aac80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800d800 tx=0x7fa0b800dcc0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.217+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1716041210 shutdown_connections 2024-05-04T12:19:20.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.217+0000 7fa0d48ce640 1 --2- 172.21.15.52:0/1716041210 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fa0b00678c0 0x7fa0b0069d80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.217+0000 7fa0d48ce640 1 --2- 172.21.15.52:0/1716041210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d015c7f0 0x7fa0d00ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.217+0000 7fa0d48ce640 1 --2- 172.21.15.52:0/1716041210 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d00cd190 0x7fa0d00aac80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.217+0000 7fa0d48ce640 1 --2- 172.21.15.52:0/1716041210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa0d00ccdb0 0x7fa0d014de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.217+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1716041210 >> 172.21.15.52:0/1716041210 conn(0x7fa0d00b14a0 msgr2=0x7fa0d00ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:20.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.217+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1716041210 shutdown_connections 2024-05-04T12:19:20.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.217+0000 7fa0d48ce640 1 -- 172.21.15.52:0/1716041210 wait complete. 2024-05-04T12:19:20.238 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-04T12:19:20.239 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-04T12:19:20.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f287325b640 1 Processor -- start 2024-05-04T12:19:20.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f287325b640 1 -- start start 2024-05-04T12:19:20.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f287325b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f286c0ad140 0x7f286c0d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f287325b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f286c0d2620 0x7f286c15fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f287325b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f286c0d2e10 con 0x7f286c15f320 2024-05-04T12:19:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f287325b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f286c15f700 con 0x7f286c0ad140 2024-05-04T12:19:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f287325b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f286c0d30f0 con 0x7f286c0d2620 2024-05-04T12:19:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f2863fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f286c0ad140 0x7f286c0d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f2870ff7640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f286c15f320 0x7f286c0aca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.52:36862/0 (socket says 172.21.15.52:36862) 2024-05-04T12:19:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f2863fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f286c0ad140 0x7f286c0d20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:37634/0 (socket says 172.21.15.52:37634) 2024-05-04T12:19:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f28717f8640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f286c0d2620 0x7f286c15fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f2870ff7640 1 -- 172.21.15.52:0/504454708 learned_addr learned my addr 172.21.15.52:0/504454708 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:20.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f2863fff640 1 -- 172.21.15.52:0/504454708 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f286c0d2620 msgr2=0x7f286c15fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:20.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f2863fff640 1 --2- 172.21.15.52:0/504454708 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f286c0d2620 0x7f286c15fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f2863fff640 1 -- 172.21.15.52:0/504454708 >> v1:172.21.15.111:6789/0 conn(0x7f286c15f320 legacy=0x7f286c0aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f2863fff640 1 -- 172.21.15.52:0/504454708 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c0d33d0 con 0x7f286c0ad140 2024-05-04T12:19:20.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.369+0000 7f28717f8640 1 --2- 172.21.15.52:0/504454708 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f286c0d2620 0x7f286c15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T12:19:20.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.373+0000 7f2863fff640 1 --2- 172.21.15.52:0/504454708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c0ad140 0x7f286c0d20e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f285c0097c0 tx=0x7f285c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aec6aed4310db9c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:20.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.373+0000 7f28637fe640 1 -- 172.21.15.52:0/504454708 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c0b0070 con 0x7f286c0ad140 2024-05-04T12:19:20.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.373+0000 7f28637fe640 1 -- 172.21.15.52:0/504454708 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285c0ab600 con 0x7f286c0ad140 2024-05-04T12:19:20.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.373+0000 7f28637fe640 1 -- 172.21.15.52:0/504454708 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c0ab9d0 con 0x7f286c0ad140 2024-05-04T12:19:20.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.373+0000 7f287325b640 1 -- 172.21.15.52:0/504454708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c0ad140 msgr2=0x7f286c0d20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.373+0000 7f287325b640 1 --2- 172.21.15.52:0/504454708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c0ad140 0x7f286c0d20e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f285c0097c0 tx=0x7f285c01c4a0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.373+0000 7f287325b640 1 -- 172.21.15.52:0/504454708 shutdown_connections 2024-05-04T12:19:20.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.373+0000 7f287325b640 1 --2- 172.21.15.52:0/504454708 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f286c0d2620 0x7f286c15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.373+0000 7f287325b640 1 --2- 172.21.15.52:0/504454708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c0ad140 0x7f286c0d20e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.373+0000 7f287325b640 1 -- 172.21.15.52:0/504454708 >> 172.21.15.52:0/504454708 conn(0x7f286c067240 msgr2=0x7f286c0d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:20.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.373+0000 7f287325b640 1 -- 172.21.15.52:0/504454708 shutdown_connections 2024-05-04T12:19:20.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.373+0000 7f287325b640 1 -- 172.21.15.52:0/504454708 wait complete. 2024-05-04T12:19:20.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f287325b640 1 Processor -- start 2024-05-04T12:19:20.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f287325b640 1 -- start start 2024-05-04T12:19:20.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f287325b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c0ad140 0x7f286c13d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f287325b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f286c0d2620 0x7f286c1fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f287325b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f286c15f320 0x7f286c0d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f287325b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f286c160e50 con 0x7f286c15f320 2024-05-04T12:19:20.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f287325b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f286c160cd0 con 0x7f286c0ad140 2024-05-04T12:19:20.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f287325b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f286c160fd0 con 0x7f286c0d2620 2024-05-04T12:19:20.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f28717f8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f286c15f320 0x7f286c0d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f28717f8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f286c15f320 0x7f286c0d3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43774/0 (socket says 172.21.15.52:43774) 2024-05-04T12:19:20.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f28717f8640 1 -- 172.21.15.52:0/4260863283 learned_addr learned my addr 172.21.15.52:0/4260863283 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:20.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f2870ff7640 1 --2- 172.21.15.52:0/4260863283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c0ad140 0x7f286c13d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f2863fff640 1 --2- 172.21.15.52:0/4260863283 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f286c0d2620 0x7f286c1fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f28717f8640 1 -- 172.21.15.52:0/4260863283 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f286c0d2620 msgr2=0x7f286c1fa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f28717f8640 1 --2- 172.21.15.52:0/4260863283 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f286c0d2620 0x7f286c1fa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f28717f8640 1 -- 172.21.15.52:0/4260863283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c0ad140 msgr2=0x7f286c13d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f28717f8640 1 --2- 172.21.15.52:0/4260863283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c0ad140 0x7f286c13d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f28717f8640 1 -- 172.21.15.52:0/4260863283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c13c510 con 0x7f286c15f320 2024-05-04T12:19:20.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f28717f8640 1 --2- 172.21.15.52:0/4260863283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f286c15f320 0x7f286c0d3e00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f286400efc0 tx=0x7f2864009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:20.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f28617fa640 1 -- 172.21.15.52:0/4260863283 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864018030 con 0x7f286c15f320 2024-05-04T12:19:20.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f28617fa640 1 -- 172.21.15.52:0/4260863283 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f286400cde0 con 0x7f286c15f320 2024-05-04T12:19:20.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f28617fa640 1 -- 172.21.15.52:0/4260863283 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864003bc0 con 0x7f286c15f320 2024-05-04T12:19:20.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f287325b640 1 -- 172.21.15.52:0/4260863283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286c13c7f0 con 0x7f286c15f320 2024-05-04T12:19:20.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.377+0000 7f287325b640 1 -- 172.21.15.52:0/4260863283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286c13cdb0 con 0x7f286c15f320 2024-05-04T12:19:20.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.381+0000 7f287325b640 1 -- 172.21.15.52:0/4260863283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286c062e50 con 0x7f286c15f320 2024-05-04T12:19:20.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.381+0000 7f28617fa640 1 -- 172.21.15.52:0/4260863283 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f286401b070 con 0x7f286c15f320 2024-05-04T12:19:20.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.381+0000 7f28617fa640 1 --2- 172.21.15.52:0/4260863283 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f28580678c0 0x7f2858069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.381+0000 7f28617fa640 1 -- 172.21.15.52:0/4260863283 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f2864086380 con 0x7f286c15f320 2024-05-04T12:19:20.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.381+0000 7f2870ff7640 1 --2- 172.21.15.52:0/4260863283 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f28580678c0 0x7f2858069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.381+0000 7f2870ff7640 1 --2- 172.21.15.52:0/4260863283 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f28580678c0 0x7f2858069d80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f286c1435f0 tx=0x7f2868009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:20.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.385+0000 7f28617fa640 1 -- 172.21.15.52:0/4260863283 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2864010040 con 0x7f286c15f320 2024-05-04T12:19:20.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.545+0000 7f287325b640 1 -- 172.21.15.52:0/4260863283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f286c075320 con 0x7f286c15f320 2024-05-04T12:19:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.621+0000 7f28617fa640 1 -- 172.21.15.52:0/4260863283 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) ==== 138+0+0 (secure 0 0 0) 0x7f2864057210 con 0x7f286c15f320 2024-05-04T12:19:20.624 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' created 2024-05-04T12:19:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.621+0000 7f287325b640 1 -- 172.21.15.52:0/4260863283 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f28580678c0 msgr2=0x7f2858069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.621+0000 7f287325b640 1 --2- 172.21.15.52:0/4260863283 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f28580678c0 0x7f2858069d80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f286c1435f0 tx=0x7f2868009040 comp rx=0 tx=0).stop 2024-05-04T12:19:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.621+0000 7f287325b640 1 -- 172.21.15.52:0/4260863283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f286c15f320 msgr2=0x7f286c0d3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.621+0000 7f287325b640 1 --2- 172.21.15.52:0/4260863283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f286c15f320 0x7f286c0d3e00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f286400efc0 tx=0x7f2864009f90 comp rx=0 tx=0).stop 2024-05-04T12:19:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.625+0000 7f287325b640 1 -- 172.21.15.52:0/4260863283 shutdown_connections 2024-05-04T12:19:20.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.625+0000 7f287325b640 1 --2- 172.21.15.52:0/4260863283 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f28580678c0 0x7f2858069d80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.625+0000 7f287325b640 1 --2- 172.21.15.52:0/4260863283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f286c15f320 0x7f286c0d3e00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.625+0000 7f287325b640 1 --2- 172.21.15.52:0/4260863283 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f286c0d2620 0x7f286c1fa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.625+0000 7f287325b640 1 --2- 172.21.15.52:0/4260863283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c0ad140 0x7f286c13d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.625+0000 7f287325b640 1 -- 172.21.15.52:0/4260863283 >> 172.21.15.52:0/4260863283 conn(0x7f286c067240 msgr2=0x7f286c0aa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:20.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.625+0000 7f287325b640 1 -- 172.21.15.52:0/4260863283 shutdown_connections 2024-05-04T12:19:20.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.625+0000 7f287325b640 1 -- 172.21.15.52:0/4260863283 wait complete. 2024-05-04T12:19:20.650 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-04T12:19:20.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.773+0000 7f46bf3bd640 1 Processor -- start 2024-05-04T12:19:20.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.773+0000 7f46bf3bd640 1 -- start start 2024-05-04T12:19:20.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46bf3bd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f46b806cc40 0x7f46b815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46bf3bd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46b806eba0 0x7f46b806d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46bf3bd640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46b806d560 con 0x7f46b815e250 2024-05-04T12:19:20.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46bf3bd640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f46b806f110 con 0x7f46b806cc40 2024-05-04T12:19:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46bf3bd640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f46b806f3f0 con 0x7f46b806eba0 2024-05-04T12:19:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46bc958640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46b806eba0 0x7f46b806d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46bd159640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f46b806cc40 0x7f46b815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46bc958640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46b806eba0 0x7f46b806d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.52:34490/0 (socket says 172.21.15.52:34490) 2024-05-04T12:19:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46bc958640 1 -- 172.21.15.52:0/1314592989 learned_addr learned my addr 172.21.15.52:0/1314592989 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46bd159640 1 -- 172.21.15.52:0/1314592989 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46b806eba0 msgr2=0x7f46b806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46bd159640 1 --2- 172.21.15.52:0/1314592989 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46b806eba0 0x7f46b806d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46bd159640 1 -- 172.21.15.52:0/1314592989 >> v1:172.21.15.111:6789/0 conn(0x7f46b815e250 legacy=0x7f46b815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46bd159640 1 -- 172.21.15.52:0/1314592989 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b806f6d0 con 0x7f46b806cc40 2024-05-04T12:19:20.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46bd159640 1 --2- 172.21.15.52:0/1314592989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46b806cc40 0x7f46b815b6e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f46b40097c0 tx=0x7f46b401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d42f8e69abc4292 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:20.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46affff640 1 -- 172.21.15.52:0/1314592989 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b40b2070 con 0x7f46b806cc40 2024-05-04T12:19:20.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46affff640 1 -- 172.21.15.52:0/1314592989 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b40ad640 con 0x7f46b806cc40 2024-05-04T12:19:20.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.777+0000 7f46affff640 1 -- 172.21.15.52:0/1314592989 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b40ada10 con 0x7f46b806cc40 2024-05-04T12:19:20.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 -- 172.21.15.52:0/1314592989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46b806cc40 msgr2=0x7f46b815b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 --2- 172.21.15.52:0/1314592989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46b806cc40 0x7f46b815b6e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f46b40097c0 tx=0x7f46b401c4a0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 -- 172.21.15.52:0/1314592989 shutdown_connections 2024-05-04T12:19:20.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 --2- 172.21.15.52:0/1314592989 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46b806eba0 0x7f46b806d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 --2- 172.21.15.52:0/1314592989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46b806cc40 0x7f46b815b6e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 -- 172.21.15.52:0/1314592989 >> 172.21.15.52:0/1314592989 conn(0x7f46b8067730 msgr2=0x7f46b80b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:20.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 -- 172.21.15.52:0/1314592989 shutdown_connections 2024-05-04T12:19:20.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 -- 172.21.15.52:0/1314592989 wait complete. 2024-05-04T12:19:20.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 Processor -- start 2024-05-04T12:19:20.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 -- start start 2024-05-04T12:19:20.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46b806cc40 0x7f46b815bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46b806eba0 0x7f46b8151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b815e250 0x7f46b814dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46b80aefd0 con 0x7f46b815e250 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81653b0 con 0x7f46b806eba0 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f46b8151e50 con 0x7f46b806cc40 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bd95a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b815e250 0x7f46b814dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bd95a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b815e250 0x7f46b814dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43778/0 (socket says 172.21.15.52:43778) 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bd95a640 1 -- 172.21.15.52:0/3644318905 learned_addr learned my addr 172.21.15.52:0/3644318905 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bc958640 1 --2- 172.21.15.52:0/3644318905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46b806eba0 0x7f46b8151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bd159640 1 --2- 172.21.15.52:0/3644318905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46b806cc40 0x7f46b815bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bd95a640 1 -- 172.21.15.52:0/3644318905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46b806cc40 msgr2=0x7f46b815bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bd95a640 1 --2- 172.21.15.52:0/3644318905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46b806cc40 0x7f46b815bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bd95a640 1 -- 172.21.15.52:0/3644318905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46b806eba0 msgr2=0x7f46b8151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bd95a640 1 --2- 172.21.15.52:0/3644318905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46b806eba0 0x7f46b8151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:20.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bd95a640 1 -- 172.21.15.52:0/3644318905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b814e4e0 con 0x7f46b815e250 2024-05-04T12:19:20.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bd95a640 1 --2- 172.21.15.52:0/3644318905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b815e250 0x7f46b814dfa0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f46b000c760 tx=0x7f46b000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:20.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46adffb640 1 -- 172.21.15.52:0/3644318905 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0003b80 con 0x7f46b815e250 2024-05-04T12:19:20.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46adffb640 1 -- 172.21.15.52:0/3644318905 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b0003d20 con 0x7f46b815e250 2024-05-04T12:19:20.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 -- 172.21.15.52:0/3644318905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b815c370 con 0x7f46b815e250 2024-05-04T12:19:20.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46adffb640 1 -- 172.21.15.52:0/3644318905 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b000aa30 con 0x7f46b815e250 2024-05-04T12:19:20.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.781+0000 7f46bf3bd640 1 -- 172.21.15.52:0/3644318905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b815c930 con 0x7f46b815e250 2024-05-04T12:19:20.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.785+0000 7f46bf3bd640 1 -- 172.21.15.52:0/3644318905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b8062e50 con 0x7f46b815e250 2024-05-04T12:19:20.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.785+0000 7f46adffb640 1 -- 172.21.15.52:0/3644318905 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f46b000abd0 con 0x7f46b815e250 2024-05-04T12:19:20.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.785+0000 7f46adffb640 1 --2- 172.21.15.52:0/3644318905 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f469c0678c0 0x7f469c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:20.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.785+0000 7f46adffb640 1 -- 172.21.15.52:0/3644318905 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5227+0+0 (secure 0 0 0) 0x7f46b00862d0 con 0x7f46b815e250 2024-05-04T12:19:20.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.785+0000 7f46bd159640 1 --2- 172.21.15.52:0/3644318905 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f469c0678c0 0x7f469c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:20.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.785+0000 7f46bd159640 1 --2- 172.21.15.52:0/3644318905 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f469c0678c0 0x7f469c069d80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f46b40af000 tx=0x7f46b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:20.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.789+0000 7f46adffb640 1 -- 172.21.15.52:0/3644318905 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f46b0057160 con 0x7f46b815e250 2024-05-04T12:19:20.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:20.953+0000 7f46bf3bd640 1 -- 172.21.15.52:0/3644318905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f46b80b29d0 con 0x7f46b815e250 2024-05-04T12:19:21.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.633+0000 7f46adffb640 1 -- 172.21.15.52:0/3644318905 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) ==== 170+0+0 (secure 0 0 0) 0x7f46b0056a60 con 0x7f46b815e250 2024-05-04T12:19:21.636 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' created 2024-05-04T12:19:21.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.637+0000 7f46bf3bd640 1 -- 172.21.15.52:0/3644318905 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f469c0678c0 msgr2=0x7f469c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:21.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.637+0000 7f46bf3bd640 1 --2- 172.21.15.52:0/3644318905 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f469c0678c0 0x7f469c069d80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f46b40af000 tx=0x7f46b4008040 comp rx=0 tx=0).stop 2024-05-04T12:19:21.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.637+0000 7f46bf3bd640 1 -- 172.21.15.52:0/3644318905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b815e250 msgr2=0x7f46b814dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:21.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.637+0000 7f46bf3bd640 1 --2- 172.21.15.52:0/3644318905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b815e250 0x7f46b814dfa0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f46b000c760 tx=0x7f46b000cc20 comp rx=0 tx=0).stop 2024-05-04T12:19:21.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.637+0000 7f46bf3bd640 1 -- 172.21.15.52:0/3644318905 shutdown_connections 2024-05-04T12:19:21.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.637+0000 7f46bf3bd640 1 --2- 172.21.15.52:0/3644318905 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f469c0678c0 0x7f469c069d80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:21.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.637+0000 7f46bf3bd640 1 --2- 172.21.15.52:0/3644318905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b815e250 0x7f46b814dfa0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:21.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.637+0000 7f46bf3bd640 1 --2- 172.21.15.52:0/3644318905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46b806eba0 0x7f46b8151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:21.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.637+0000 7f46bf3bd640 1 --2- 172.21.15.52:0/3644318905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46b806cc40 0x7f46b815bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:21.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.637+0000 7f46bf3bd640 1 -- 172.21.15.52:0/3644318905 >> 172.21.15.52:0/3644318905 conn(0x7f46b8067730 msgr2=0x7f46b815ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:21.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.641+0000 7f46bf3bd640 1 -- 172.21.15.52:0/3644318905 shutdown_connections 2024-05-04T12:19:21.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.641+0000 7f46bf3bd640 1 -- 172.21.15.52:0/3644318905 wait complete. 2024-05-04T12:19:21.680 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-04T12:19:21.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc770ac7640 1 Processor -- start 2024-05-04T12:19:21.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc770ac7640 1 -- start start 2024-05-04T12:19:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc770ac7640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc76c06cc40 0x7fc76c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc770ac7640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc76c06eba0 0x7fc76c06d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc770ac7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc76c15e250 0x7fc76c15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc770ac7640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc76c06d560 con 0x7fc76c06cc40 2024-05-04T12:19:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc770ac7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fc76c06f110 con 0x7fc76c15e250 2024-05-04T12:19:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc770ac7640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc76c06f3f0 con 0x7fc76c06eba0 2024-05-04T12:19:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc76a59c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc76c06cc40 0x7fc76c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc76a59c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc76c06cc40 0x7fc76c15b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43780/0 (socket says 172.21.15.52:43780) 2024-05-04T12:19:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc76a59c640 1 -- 172.21.15.52:0/213252155 learned_addr learned my addr 172.21.15.52:0/213252155 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc76ad9d640 1 --2- 172.21.15.52:0/213252155 >> v2:172.21.15.111:3300/0 conn(0x7fc76c15e250 0x7fc76c15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc769d9b640 1 --2- 172.21.15.52:0/213252155 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc76c06eba0 0x7fc76c06d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc76a59c640 1 -- 172.21.15.52:0/213252155 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc76c06eba0 msgr2=0x7fc76c06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:21.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc76a59c640 1 --2- 172.21.15.52:0/213252155 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc76c06eba0 0x7fc76c06d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:21.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc76a59c640 1 -- 172.21.15.52:0/213252155 >> v2:172.21.15.111:3300/0 conn(0x7fc76c15e250 msgr2=0x7fc76c15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:21.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc76a59c640 1 --2- 172.21.15.52:0/213252155 >> v2:172.21.15.111:3300/0 conn(0x7fc76c15e250 0x7fc76c15e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:21.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc76a59c640 1 -- 172.21.15.52:0/213252155 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c06f6d0 con 0x7fc76c06cc40 2024-05-04T12:19:21.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc76ad9d640 1 --2- 172.21.15.52:0/213252155 >> v2:172.21.15.111:3300/0 conn(0x7fc76c15e250 0x7fc76c15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:21.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.801+0000 7fc769d9b640 1 --2- 172.21.15.52:0/213252155 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc76c06eba0 0x7fc76c06d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:21.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc76a59c640 1 --2- 172.21.15.52:0/213252155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c06cc40 0x7fc76c15b6e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc76c072520 tx=0x7fc76401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e999692b872d6ad3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:21.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc76959a640 1 -- 172.21.15.52:0/213252155 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7640a56e0 con 0x7fc76c06cc40 2024-05-04T12:19:21.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc76959a640 1 -- 172.21.15.52:0/213252155 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7640a5880 con 0x7fc76c06cc40 2024-05-04T12:19:21.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc76959a640 1 -- 172.21.15.52:0/213252155 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7640a5c50 con 0x7fc76c06cc40 2024-05-04T12:19:21.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 -- 172.21.15.52:0/213252155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c06cc40 msgr2=0x7fc76c15b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:21.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 --2- 172.21.15.52:0/213252155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c06cc40 0x7fc76c15b6e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc76c072520 tx=0x7fc76401c9f0 comp rx=0 tx=0).stop 2024-05-04T12:19:21.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 -- 172.21.15.52:0/213252155 shutdown_connections 2024-05-04T12:19:21.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 --2- 172.21.15.52:0/213252155 >> v2:172.21.15.111:3300/0 conn(0x7fc76c15e250 0x7fc76c15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:21.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 --2- 172.21.15.52:0/213252155 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc76c06eba0 0x7fc76c06d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:21.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 --2- 172.21.15.52:0/213252155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c06cc40 0x7fc76c15b6e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:21.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 -- 172.21.15.52:0/213252155 >> 172.21.15.52:0/213252155 conn(0x7fc76c067730 msgr2=0x7fc76c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:21.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 -- 172.21.15.52:0/213252155 shutdown_connections 2024-05-04T12:19:21.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 -- 172.21.15.52:0/213252155 wait complete. 2024-05-04T12:19:21.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 Processor -- start 2024-05-04T12:19:21.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 -- start start 2024-05-04T12:19:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c06cc40 0x7fc76c0afa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc76c06eba0 0x7fc76c14c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc76c15e250 0x7fc76c140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c0aefd0 con 0x7fc76c06cc40 2024-05-04T12:19:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c0cc410 con 0x7fc76c06eba0 2024-05-04T12:19:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.805+0000 7fc770ac7640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc76c0b0120 con 0x7fc76c15e250 2024-05-04T12:19:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc76a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c06cc40 0x7fc76c0afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc76a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c06cc40 0x7fc76c0afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43790/0 (socket says 172.21.15.52:43790) 2024-05-04T12:19:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc76a59c640 1 -- 172.21.15.52:0/3289847986 learned_addr learned my addr 172.21.15.52:0/3289847986 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc769d9b640 1 --2- 172.21.15.52:0/3289847986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc76c06eba0 0x7fc76c14c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc76ad9d640 1 --2- 172.21.15.52:0/3289847986 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc76c15e250 0x7fc76c140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc76a59c640 1 -- 172.21.15.52:0/3289847986 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc76c15e250 msgr2=0x7fc76c140910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc76a59c640 1 --2- 172.21.15.52:0/3289847986 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc76c15e250 0x7fc76c140910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc76a59c640 1 -- 172.21.15.52:0/3289847986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc76c06eba0 msgr2=0x7fc76c14c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc76a59c640 1 --2- 172.21.15.52:0/3289847986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc76c06eba0 0x7fc76c14c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc76a59c640 1 -- 172.21.15.52:0/3289847986 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c0b0400 con 0x7fc76c06cc40 2024-05-04T12:19:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc76a59c640 1 --2- 172.21.15.52:0/3289847986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c06cc40 0x7fc76c0afa90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc7640af040 tx=0x7fc764019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc762ffd640 1 -- 172.21.15.52:0/3289847986 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7640b8670 con 0x7fc76c06cc40 2024-05-04T12:19:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc762ffd640 1 -- 172.21.15.52:0/3289847986 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7640be070 con 0x7fc76c06cc40 2024-05-04T12:19:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc762ffd640 1 -- 172.21.15.52:0/3289847986 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7640b8a90 con 0x7fc76c06cc40 2024-05-04T12:19:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc770ac7640 1 -- 172.21.15.52:0/3289847986 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76c0b06e0 con 0x7fc76c06cc40 2024-05-04T12:19:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc770ac7640 1 -- 172.21.15.52:0/3289847986 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76c0b0ca0 con 0x7fc76c06cc40 2024-05-04T12:19:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc770ac7640 1 -- 172.21.15.52:0/3289847986 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc76c062e50 con 0x7fc76c06cc40 2024-05-04T12:19:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc762ffd640 1 -- 172.21.15.52:0/3289847986 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc7640b8c30 con 0x7fc76c06cc40 2024-05-04T12:19:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc762ffd640 1 --2- 172.21.15.52:0/3289847986 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc74c0678c0 0x7fc74c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:21.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc769d9b640 1 --2- 172.21.15.52:0/3289847986 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc74c0678c0 0x7fc74c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:21.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc762ffd640 1 -- 172.21.15.52:0/3289847986 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5609+0+0 (secure 0 0 0) 0x7fc764120e90 con 0x7fc76c06cc40 2024-05-04T12:19:21.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.809+0000 7fc769d9b640 1 --2- 172.21.15.52:0/3289847986 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc74c0678c0 0x7fc74c069d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc76c1fac90 tx=0x7fc754008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:21.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:21.817+0000 7fc762ffd640 1 -- 172.21.15.52:0/3289847986 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc7640165e0 con 0x7fc76c06cc40 2024-05-04T12:19:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.009+0000 7fc770ac7640 1 -- 172.21.15.52:0/3289847986 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc76c0cf870 con 0x7fc76c06cc40 2024-05-04T12:19:22.545 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-05-04T12:19:22.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.669+0000 7fc762ffd640 1 -- 172.21.15.52:0/3289847986 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7fc7640f1ac0 con 0x7fc76c06cc40 2024-05-04T12:19:22.674 INFO:teuthology.orchestra.run.smithi052.stderr:new fs with metadata pool 4 and data pool 5 2024-05-04T12:19:22.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.677+0000 7fc770ac7640 1 -- 172.21.15.52:0/3289847986 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc74c0678c0 msgr2=0x7fc74c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:22.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.677+0000 7fc770ac7640 1 --2- 172.21.15.52:0/3289847986 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc74c0678c0 0x7fc74c069d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc76c1fac90 tx=0x7fc754008040 comp rx=0 tx=0).stop 2024-05-04T12:19:22.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.677+0000 7fc770ac7640 1 -- 172.21.15.52:0/3289847986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c06cc40 msgr2=0x7fc76c0afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:22.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.677+0000 7fc770ac7640 1 --2- 172.21.15.52:0/3289847986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c06cc40 0x7fc76c0afa90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc7640af040 tx=0x7fc764019790 comp rx=0 tx=0).stop 2024-05-04T12:19:22.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.681+0000 7fc770ac7640 1 -- 172.21.15.52:0/3289847986 shutdown_connections 2024-05-04T12:19:22.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.681+0000 7fc770ac7640 1 --2- 172.21.15.52:0/3289847986 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc74c0678c0 0x7fc74c069d80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:22.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.681+0000 7fc770ac7640 1 --2- 172.21.15.52:0/3289847986 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc76c15e250 0x7fc76c140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:22.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.681+0000 7fc770ac7640 1 --2- 172.21.15.52:0/3289847986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc76c06eba0 0x7fc76c14c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:22.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.681+0000 7fc770ac7640 1 --2- 172.21.15.52:0/3289847986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c06cc40 0x7fc76c0afa90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:22.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.681+0000 7fc770ac7640 1 -- 172.21.15.52:0/3289847986 >> 172.21.15.52:0/3289847986 conn(0x7fc76c067730 msgr2=0x7fc76c0ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:22.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.681+0000 7fc770ac7640 1 -- 172.21.15.52:0/3289847986 shutdown_connections 2024-05-04T12:19:22.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.681+0000 7fc770ac7640 1 -- 172.21.15.52:0/3289847986 wait complete. 2024-05-04T12:19:22.715 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:19:22.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaf078e640 1 Processor -- start 2024-05-04T12:19:22.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaf078e640 1 -- start start 2024-05-04T12:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaf078e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faae8147940 0x7faae81621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaf078e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7faae815f6a0 0x7faae80cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaf078e640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faae815fb30 con 0x7faae80d03d0 2024-05-04T12:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaf078e640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7faae815fef0 con 0x7faae815f6a0 2024-05-04T12:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaf078e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7faae80d0810 con 0x7faae8147940 2024-05-04T12:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaeed2b640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7faae80d03d0 0x7faae8070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:37776/0 (socket says 172.21.15.52:37776) 2024-05-04T12:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaeed2b640 1 -- 172.21.15.52:0/3517091368 learned_addr learned my addr 172.21.15.52:0/3517091368 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaedd29640 1 --2- 172.21.15.52:0/3517091368 >> v2:172.21.15.52:3300/0 conn(0x7faae815f6a0 0x7faae80cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaee52a640 1 --2- 172.21.15.52:0/3517091368 >> v2:172.21.15.111:3300/0 conn(0x7faae8147940 0x7faae81621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaedd29640 1 -- 172.21.15.52:0/3517091368 >> v2:172.21.15.111:3300/0 conn(0x7faae8147940 msgr2=0x7faae81621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaedd29640 1 --2- 172.21.15.52:0/3517091368 >> v2:172.21.15.111:3300/0 conn(0x7faae8147940 0x7faae81621f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaedd29640 1 -- 172.21.15.52:0/3517091368 >> v1:172.21.15.52:6789/0 conn(0x7faae80d03d0 legacy=0x7faae8070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaedd29640 1 -- 172.21.15.52:0/3517091368 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae81627d0 con 0x7faae815f6a0 2024-05-04T12:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.829+0000 7faaed528640 1 -- 172.21.15.52:0/3517091368 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1970371461 0 0) 0x7faae815fb30 con 0x7faae80d03d0 2024-05-04T12:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaedd29640 1 --2- 172.21.15.52:0/3517091368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faae815f6a0 0x7faae80cfe90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7faae80d0ce0 tx=0x7faad801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609c5e99719d4e91 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaed528640 1 -- 172.21.15.52:0/3517091368 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad80b0070 con 0x7faae815f6a0 2024-05-04T12:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaed528640 1 -- 172.21.15.52:0/3517091368 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faad80ab600 con 0x7faae815f6a0 2024-05-04T12:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaed528640 1 -- 172.21.15.52:0/3517091368 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad80ab9d0 con 0x7faae815f6a0 2024-05-04T12:19:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 -- 172.21.15.52:0/3517091368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faae815f6a0 msgr2=0x7faae80cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 --2- 172.21.15.52:0/3517091368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faae815f6a0 0x7faae80cfe90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7faae80d0ce0 tx=0x7faad801c480 comp rx=0 tx=0).stop 2024-05-04T12:19:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 -- 172.21.15.52:0/3517091368 shutdown_connections 2024-05-04T12:19:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 --2- 172.21.15.52:0/3517091368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faae815f6a0 0x7faae80cfe90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:22.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 --2- 172.21.15.52:0/3517091368 >> v2:172.21.15.111:3300/0 conn(0x7faae8147940 0x7faae81621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:22.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 -- 172.21.15.52:0/3517091368 >> 172.21.15.52:0/3517091368 conn(0x7faae80d0f80 msgr2=0x7faae80d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:22.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 -- 172.21.15.52:0/3517091368 shutdown_connections 2024-05-04T12:19:22.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 -- 172.21.15.52:0/3517091368 wait complete. 2024-05-04T12:19:22.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 Processor -- start 2024-05-04T12:19:22.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 -- start start 2024-05-04T12:19:22.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faae80d03d0 0x7faae815c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:22.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae8147940 0x7faae81f7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:22.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faae815f6a0 0x7faae8157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:22.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faae816b710 con 0x7faae80d03d0 2024-05-04T12:19:22.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faae816b590 con 0x7faae8147940 2024-05-04T12:19:22.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaf078e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7faae816b890 con 0x7faae815f6a0 2024-05-04T12:19:22.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaee52a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faae80d03d0 0x7faae815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:22.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaee52a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faae80d03d0 0x7faae815c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43798/0 (socket says 172.21.15.52:43798) 2024-05-04T12:19:22.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaee52a640 1 -- 172.21.15.52:0/2614185582 learned_addr learned my addr 172.21.15.52:0/2614185582 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:22.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaedd29640 1 --2- 172.21.15.52:0/2614185582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae8147940 0x7faae81f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:22.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaeed2b640 1 --2- 172.21.15.52:0/2614185582 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faae815f6a0 0x7faae8157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:22.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaee52a640 1 -- 172.21.15.52:0/2614185582 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faae815f6a0 msgr2=0x7faae8157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:22.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaee52a640 1 --2- 172.21.15.52:0/2614185582 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faae815f6a0 0x7faae8157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:22.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaee52a640 1 -- 172.21.15.52:0/2614185582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae8147940 msgr2=0x7faae81f7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:22.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaee52a640 1 --2- 172.21.15.52:0/2614185582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae8147940 0x7faae81f7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:22.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaee52a640 1 -- 172.21.15.52:0/2614185582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae81621d0 con 0x7faae80d03d0 2024-05-04T12:19:22.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.833+0000 7faaee52a640 1 --2- 172.21.15.52:0/2614185582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faae80d03d0 0x7faae815c260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7faae000efc0 tx=0x7faae000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:22.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.837+0000 7faadeffd640 1 -- 172.21.15.52:0/2614185582 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0018020 con 0x7faae80d03d0 2024-05-04T12:19:22.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.837+0000 7faadeffd640 1 -- 172.21.15.52:0/2614185582 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae0014480 con 0x7faae80d03d0 2024-05-04T12:19:22.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.837+0000 7faaf078e640 1 -- 172.21.15.52:0/2614185582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae8159670 con 0x7faae80d03d0 2024-05-04T12:19:22.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.837+0000 7faadeffd640 1 -- 172.21.15.52:0/2614185582 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0014850 con 0x7faae80d03d0 2024-05-04T12:19:22.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.837+0000 7faaf078e640 1 -- 172.21.15.52:0/2614185582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae8159c30 con 0x7faae80d03d0 2024-05-04T12:19:22.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.837+0000 7faaf078e640 1 -- 172.21.15.52:0/2614185582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae8062e50 con 0x7faae80d03d0 2024-05-04T12:19:22.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.837+0000 7faadeffd640 1 -- 172.21.15.52:0/2614185582 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faae001b020 con 0x7faae80d03d0 2024-05-04T12:19:22.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.837+0000 7faadeffd640 1 --2- 172.21.15.52:0/2614185582 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faacc0678c0 0x7faacc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:22.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.837+0000 7faaedd29640 1 --2- 172.21.15.52:0/2614185582 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faacc0678c0 0x7faacc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:22.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.837+0000 7faadeffd640 1 -- 172.21.15.52:0/2614185582 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7faae0086af0 con 0x7faae80d03d0 2024-05-04T12:19:22.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.837+0000 7faaedd29640 1 --2- 172.21.15.52:0/2614185582 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faacc0678c0 0x7faacc069d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faad80ad000 tx=0x7faad80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:22.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:22.841+0000 7faadeffd640 1 -- 172.21.15.52:0/2614185582 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faae0022030 con 0x7faae80d03d0 2024-05-04T12:19:23.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.005+0000 7faaf078e640 1 -- 172.21.15.52:0/2614185582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faae8075320 con 0x7faae80d03d0 2024-05-04T12:19:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.009+0000 7faadeffd640 1 -- 172.21.15.52:0/2614185582 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16694 (secure 0 0 0) 0x7faae0010040 con 0x7faae80d03d0 2024-05-04T12:19:23.011 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:23.011 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":33,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:19:22.017127+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:19:20.589701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-04T12:19:20.997186+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:19:23.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.009+0000 7faaf078e640 1 -- 172.21.15.52:0/2614185582 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faacc0678c0 msgr2=0x7faacc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:23.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.009+0000 7faaf078e640 1 --2- 172.21.15.52:0/2614185582 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faacc0678c0 0x7faacc069d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faad80ad000 tx=0x7faad80a6000 comp rx=0 tx=0).stop 2024-05-04T12:19:23.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.009+0000 7faaf078e640 1 -- 172.21.15.52:0/2614185582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faae80d03d0 msgr2=0x7faae815c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:23.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.009+0000 7faaf078e640 1 --2- 172.21.15.52:0/2614185582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faae80d03d0 0x7faae815c260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7faae000efc0 tx=0x7faae000c490 comp rx=0 tx=0).stop 2024-05-04T12:19:23.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.009+0000 7faaf078e640 1 -- 172.21.15.52:0/2614185582 shutdown_connections 2024-05-04T12:19:23.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.009+0000 7faaf078e640 1 --2- 172.21.15.52:0/2614185582 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faacc0678c0 0x7faacc069d80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.009+0000 7faaf078e640 1 --2- 172.21.15.52:0/2614185582 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faae815f6a0 0x7faae8157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.009+0000 7faaf078e640 1 --2- 172.21.15.52:0/2614185582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae8147940 0x7faae81f7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.009+0000 7faaf078e640 1 --2- 172.21.15.52:0/2614185582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faae80d03d0 0x7faae815c260 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.009+0000 7faaf078e640 1 -- 172.21.15.52:0/2614185582 >> 172.21.15.52:0/2614185582 conn(0x7faae80d0f80 msgr2=0x7faae80cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:23.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.009+0000 7faaf078e640 1 -- 172.21.15.52:0/2614185582 shutdown_connections 2024-05-04T12:19:23.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.009+0000 7faaf078e640 1 -- 172.21.15.52:0/2614185582 wait complete. 2024-05-04T12:19:23.031 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:19:23.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.153+0000 7f2cc1094640 1 Processor -- start 2024-05-04T12:19:23.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cc1094640 1 -- start start 2024-05-04T12:19:23.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cc1094640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2cbc1441b0 0x7f2cbc1627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:23.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cc1094640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2cbc0b6bf0 0x7f2cbc141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:23.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cc1094640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cbc162cf0 con 0x7f2cbc144590 2024-05-04T12:19:23.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cc1094640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f2cbc15f6d0 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cc1094640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f2cbc15eac0 con 0x7f2cbc1441b0 2024-05-04T12:19:23.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cb259c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f2cbc144590 0x7f2cbc0b64e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:37782/0 (socket says 172.21.15.52:37782) 2024-05-04T12:19:23.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cb259c640 1 -- 172.21.15.52:0/3940676835 learned_addr learned my addr 172.21.15.52:0/3940676835 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:23.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cbb59e640 1 --2- 172.21.15.52:0/3940676835 >> v2:172.21.15.52:3300/0 conn(0x7f2cbc0b6bf0 0x7f2cbc141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:23.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cbad9d640 1 --2- 172.21.15.52:0/3940676835 >> v2:172.21.15.111:3300/0 conn(0x7f2cbc1441b0 0x7f2cbc1627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:23.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cba59c640 1 -- 172.21.15.52:0/3940676835 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 306861610 0 0) 0x7f2ca4001120 con 0x7f2cbc144590 2024-05-04T12:19:23.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cba59c640 1 -- 172.21.15.52:0/3940676835 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ca8003530 con 0x7f2cbc144590 2024-05-04T12:19:23.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cbb59e640 1 -- 172.21.15.52:0/3940676835 >> v2:172.21.15.111:3300/0 conn(0x7f2cbc1441b0 msgr2=0x7f2cbc1627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:23.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cbb59e640 1 --2- 172.21.15.52:0/3940676835 >> v2:172.21.15.111:3300/0 conn(0x7f2cbc1441b0 0x7f2cbc1627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cbb59e640 1 -- 172.21.15.52:0/3940676835 >> v1:172.21.15.52:6789/0 conn(0x7f2cbc144590 legacy=0x7f2cbc0b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:23.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cbb59e640 1 -- 172.21.15.52:0/3940676835 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc15eda0 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.157+0000 7f2cbad9d640 1 --2- 172.21.15.52:0/3940676835 >> v2:172.21.15.111:3300/0 conn(0x7f2cbc1441b0 0x7f2cbc1627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:23.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cbb59e640 1 --2- 172.21.15.52:0/3940676835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc0b6bf0 0x7f2cbc141ab0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc070d70 tx=0x7f2cb401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2dc746da3510885 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:23.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cba59c640 1 -- 172.21.15.52:0/3940676835 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40b2070 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cba59c640 1 -- 172.21.15.52:0/3940676835 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb40a59c0 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cba59c640 1 -- 172.21.15.52:0/3940676835 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40a5d90 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 -- 172.21.15.52:0/3940676835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc0b6bf0 msgr2=0x7f2cbc141ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:23.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 --2- 172.21.15.52:0/3940676835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc0b6bf0 0x7f2cbc141ab0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc070d70 tx=0x7f2cb401c900 comp rx=0 tx=0).stop 2024-05-04T12:19:23.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 -- 172.21.15.52:0/3940676835 shutdown_connections 2024-05-04T12:19:23.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 --2- 172.21.15.52:0/3940676835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc0b6bf0 0x7f2cbc141ab0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 --2- 172.21.15.52:0/3940676835 >> v2:172.21.15.111:3300/0 conn(0x7f2cbc1441b0 0x7f2cbc1627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 -- 172.21.15.52:0/3940676835 >> 172.21.15.52:0/3940676835 conn(0x7f2cbc0b52d0 msgr2=0x7f2cbc0b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:23.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 -- 172.21.15.52:0/3940676835 shutdown_connections 2024-05-04T12:19:23.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 -- 172.21.15.52:0/3940676835 wait complete. 2024-05-04T12:19:23.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 Processor -- start 2024-05-04T12:19:23.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 -- start start 2024-05-04T12:19:23.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc0b6bf0 0x7f2cbc153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:23.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2cbc1441b0 0x7f2cbc071c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:23.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cbc144590 0x7f2cbc160e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:23.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc143c10 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc143a90 con 0x7f2cbc144590 2024-05-04T12:19:23.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cc1094640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f2cbc142800 con 0x7f2cbc1441b0 2024-05-04T12:19:23.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.161+0000 7f2cbad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc0b6bf0 0x7f2cbc153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:23.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cbad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc0b6bf0 0x7f2cbc153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43818/0 (socket says 172.21.15.52:43818) 2024-05-04T12:19:23.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cbad9d640 1 -- 172.21.15.52:0/3241443878 learned_addr learned my addr 172.21.15.52:0/3241443878 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:23.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cb259c640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2cbc1441b0 0x7f2cbc071c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:23.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cbb59e640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cbc144590 0x7f2cbc160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:23.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cbad9d640 1 -- 172.21.15.52:0/3241443878 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2cbc1441b0 msgr2=0x7f2cbc071c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:23.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cbad9d640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2cbc1441b0 0x7f2cbc071c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cbad9d640 1 -- 172.21.15.52:0/3241443878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cbc144590 msgr2=0x7f2cbc160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:23.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cbad9d640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cbc144590 0x7f2cbc160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cbad9d640 1 -- 172.21.15.52:0/3241443878 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc158200 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cb259c640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2cbc1441b0 0x7f2cbc071c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:23.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cbb59e640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cbc144590 0x7f2cbc160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:23.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cbad9d640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc0b6bf0 0x7f2cbc153ef0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00e840 tx=0x7f2cac00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:23.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cb3fff640 1 -- 172.21.15.52:0/3241443878 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac017070 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cb3fff640 1 -- 172.21.15.52:0/3241443878 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cac00c8d0 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cc1094640 1 -- 172.21.15.52:0/3241443878 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cbc13b7d0 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cb3fff640 1 -- 172.21.15.52:0/3241443878 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00f040 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cc1094640 1 -- 172.21.15.52:0/3241443878 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cbc154430 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cc1094640 1 -- 172.21.15.52:0/3241443878 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cbc13b5f0 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cb3fff640 1 -- 172.21.15.52:0/3241443878 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2cac01a070 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cb3fff640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2c9c0678f0 0x7f2c9c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:23.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cb3fff640 1 -- 172.21.15.52:0/3241443878 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2cac0859d0 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cb259c640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2c9c0678f0 0x7f2c9c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:23.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.165+0000 7f2cb259c640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2c9c0678f0 0x7f2c9c069db0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc15ff00 tx=0x7f2ca4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:23.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.169+0000 7f2cb3fff640 1 -- 172.21.15.52:0/3241443878 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2cbc13b5f0 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.333+0000 7f2cc1094640 1 -- 172.21.15.52:0/3241443878 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cbc062e50 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.337+0000 7f2cb3fff640 1 -- 172.21.15.52:0/3241443878 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16694 (secure 0 0 0) 0x7f2cac0563e0 con 0x7f2cbc0b6bf0 2024-05-04T12:19:23.341 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:23.341 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":33,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:19:22.017127+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:19:20.589701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-04T12:19:20.997186+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:19:23.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.341+0000 7f2cc1094640 1 -- 172.21.15.52:0/3241443878 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2c9c0678f0 msgr2=0x7f2c9c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:23.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.341+0000 7f2cc1094640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2c9c0678f0 0x7f2c9c069db0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc15ff00 tx=0x7f2ca4009040 comp rx=0 tx=0).stop 2024-05-04T12:19:23.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.341+0000 7f2cc1094640 1 -- 172.21.15.52:0/3241443878 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc0b6bf0 msgr2=0x7f2cbc153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:23.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.341+0000 7f2cc1094640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc0b6bf0 0x7f2cbc153ef0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00e840 tx=0x7f2cac00ed00 comp rx=0 tx=0).stop 2024-05-04T12:19:23.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.341+0000 7f2cc1094640 1 -- 172.21.15.52:0/3241443878 shutdown_connections 2024-05-04T12:19:23.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.341+0000 7f2cc1094640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f2c9c0678f0 0x7f2c9c069db0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.341+0000 7f2cc1094640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cbc144590 0x7f2cbc160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.341+0000 7f2cc1094640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2cbc1441b0 0x7f2cbc071c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.341+0000 7f2cc1094640 1 --2- 172.21.15.52:0/3241443878 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc0b6bf0 0x7f2cbc153ef0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.341+0000 7f2cc1094640 1 -- 172.21.15.52:0/3241443878 >> 172.21.15.52:0/3241443878 conn(0x7f2cbc0b52d0 msgr2=0x7f2cbc0d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:23.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.341+0000 7f2cc1094640 1 -- 172.21.15.52:0/3241443878 shutdown_connections 2024-05-04T12:19:23.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.341+0000 7f2cc1094640 1 -- 172.21.15.52:0/3241443878 wait complete. 2024-05-04T12:19:23.361 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-04T12:19:23.479 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:19:23.481 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:19:23.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b5a9f8640 1 Processor -- start 2024-05-04T12:19:23.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b5a9f8640 1 -- start start 2024-05-04T12:19:23.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b5a9f8640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b54070de0 0x7f9b540c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:23.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b5a9f8640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b5415c6c0 con 0x7f9b540c52b0 2024-05-04T12:19:23.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b5a9f8640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b5415c9a0 con 0x7f9b540c4630 2024-05-04T12:19:23.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b5a9f8640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f9b540c5a80 con 0x7f9b54070de0 2024-05-04T12:19:23.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b58f95640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f9b540c52b0 0x7f9b5415fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:37796/0 (socket says 172.21.15.52:37796) 2024-05-04T12:19:23.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b58f95640 1 -- 172.21.15.52:0/3494986740 learned_addr learned my addr 172.21.15.52:0/3494986740 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:23.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b537fe640 1 --2- 172.21.15.52:0/3494986740 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b54070de0 0x7f9b540c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:23.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1245538735 0 0) 0x7f9b5415c6c0 con 0x7f9b540c52b0 2024-05-04T12:19:23.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b40003530 con 0x7f9b540c52b0 2024-05-04T12:19:23.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2330641245 0 0) 0x7f9b5415c9a0 con 0x7f9b540c4630 2024-05-04T12:19:23.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b5415c6c0 con 0x7f9b540c4630 2024-05-04T12:19:23.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4282265471 0 0) 0x7f9b40003530 con 0x7f9b540c52b0 2024-05-04T12:19:23.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b54070de0 msgr2=0x7f9b540c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:23.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 --2- 172.21.15.52:0/3494986740 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b54070de0 0x7f9b540c4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 >> v1:172.21.15.111:6789/0 conn(0x7f9b540c4630 legacy=0x7f9b540706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:23.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5415cc80 con 0x7f9b540c52b0 2024-05-04T12:19:23.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f9b44001600 con 0x7f9b540c52b0 2024-05-04T12:19:23.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 >> v1:172.21.15.52:6789/0 conn(0x7f9b540c52b0 legacy=0x7f9b5415fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:23.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 --2- 172.21.15.52:0/3494986740 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b54070de0 0x7f9b40003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:23.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 --2- 172.21.15.52:0/3494986740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40004130 0x7f9b40004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:23.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 --2- 172.21.15.52:0/3494986740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b40004d50 0x7f9b400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:23.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9b540c5a80 con 0x7f9b40004d50 2024-05-04T12:19:23.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4000bb10 con 0x7f9b40004130 2024-05-04T12:19:23.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f9b4000bd10 con 0x7f9b54070de0 2024-05-04T12:19:23.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b58f95640 1 --2- 172.21.15.52:0/3494986740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b40004d50 0x7f9b400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:23.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b53fff640 1 --2- 172.21.15.52:0/3494986740 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b54070de0 0x7f9b40003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:23.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b537fe640 1 --2- 172.21.15.52:0/3494986740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40004130 0x7f9b40004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:23.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b58f95640 1 -- 172.21.15.52:0/3494986740 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b54070de0 msgr2=0x7f9b40003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:23.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b58f95640 1 --2- 172.21.15.52:0/3494986740 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b54070de0 0x7f9b40003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b58f95640 1 -- 172.21.15.52:0/3494986740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40004130 msgr2=0x7f9b40004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:23.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b58f95640 1 --2- 172.21.15.52:0/3494986740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40004130 0x7f9b40004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b58f95640 1 -- 172.21.15.52:0/3494986740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b40003530 con 0x7f9b40004d50 2024-05-04T12:19:23.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.481+0000 7f9b537fe640 1 --2- 172.21.15.52:0/3494986740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40004130 0x7f9b40004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:23.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.485+0000 7f9b58f95640 1 --2- 172.21.15.52:0/3494986740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b40004d50 0x7f9b400051d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9b44007f50 tx=0x7f9b4401f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c88906181652a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:23.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.485+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44007b70 con 0x7f9b40004d50 2024-05-04T12:19:23.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.485+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b440ac020 con 0x7f9b40004d50 2024-05-04T12:19:23.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.485+0000 7f9b52ffd640 1 -- 172.21.15.52:0/3494986740 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b440058e0 con 0x7f9b40004d50 2024-05-04T12:19:23.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.485+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/3494986740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b40004d50 msgr2=0x7f9b400051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:23.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.485+0000 7f9b5a9f8640 1 --2- 172.21.15.52:0/3494986740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b40004d50 0x7f9b400051d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9b44007f50 tx=0x7f9b4401f1f0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.485+0000 7f9b58f95640 1 -- 172.21.15.52:0/3494986740 reap_dead start 2024-05-04T12:19:23.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.485+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/3494986740 shutdown_connections 2024-05-04T12:19:23.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.485+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/3494986740 >> 172.21.15.52:0/3494986740 conn(0x7f9b540d12c0 msgr2=0x7f9b540d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:23.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.485+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/3494986740 shutdown_connections 2024-05-04T12:19:23.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.485+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/3494986740 wait complete. 2024-05-04T12:19:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b5a9f8640 1 Processor -- start 2024-05-04T12:19:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b5a9f8640 1 -- start start 2024-05-04T12:19:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b5a9f8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40004d50 0x7f9b540cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b5a9f8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b54150e60 0x7f9b54145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b5a9f8640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b5415e630 0x7f9b5415eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b5a9f8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5415cff0 con 0x7f9b54150e60 2024-05-04T12:19:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b5a9f8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5415c470 con 0x7f9b40004d50 2024-05-04T12:19:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b5a9f8640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f9b540c4a10 con 0x7f9b5415e630 2024-05-04T12:19:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b537fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b54150e60 0x7f9b54145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b537fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b54150e60 0x7f9b54145d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43846/0 (socket says 172.21.15.52:43846) 2024-05-04T12:19:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b537fe640 1 -- 172.21.15.52:0/930741497 learned_addr learned my addr 172.21.15.52:0/930741497 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b53fff640 1 --2- 172.21.15.52:0/930741497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40004d50 0x7f9b540cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b58f95640 1 --2- 172.21.15.52:0/930741497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b5415e630 0x7f9b5415eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b537fe640 1 -- 172.21.15.52:0/930741497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b5415e630 msgr2=0x7f9b5415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b537fe640 1 --2- 172.21.15.52:0/930741497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b5415e630 0x7f9b5415eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b537fe640 1 -- 172.21.15.52:0/930741497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40004d50 msgr2=0x7f9b540cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b537fe640 1 --2- 172.21.15.52:0/930741497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40004d50 0x7f9b540cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b537fe640 1 -- 172.21.15.52:0/930741497 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b541462b0 con 0x7f9b54150e60 2024-05-04T12:19:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b537fe640 1 --2- 172.21.15.52:0/930741497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b54150e60 0x7f9b54145d70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00f350 tx=0x7f9b3c00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b50ff9640 1 -- 172.21.15.52:0/930741497 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c0039a0 con 0x7f9b54150e60 2024-05-04T12:19:23.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b50ff9640 1 -- 172.21.15.52:0/930741497 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b3c003b40 con 0x7f9b54150e60 2024-05-04T12:19:23.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/930741497 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b540ce1d0 con 0x7f9b54150e60 2024-05-04T12:19:23.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b50ff9640 1 -- 172.21.15.52:0/930741497 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c004020 con 0x7f9b54150e60 2024-05-04T12:19:23.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/930741497 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b540ce790 con 0x7f9b54150e60 2024-05-04T12:19:23.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/930741497 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b54062e50 con 0x7f9b54150e60 2024-05-04T12:19:23.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.489+0000 7f9b50ff9640 1 -- 172.21.15.52:0/930741497 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9b3c0041c0 con 0x7f9b54150e60 2024-05-04T12:19:23.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.493+0000 7f9b50ff9640 1 --2- 172.21.15.52:0/930741497 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f9b340678f0 0x7f9b34069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:23.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.493+0000 7f9b53fff640 1 --2- 172.21.15.52:0/930741497 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f9b340678f0 0x7f9b34069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:23.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.493+0000 7f9b50ff9640 1 -- 172.21.15.52:0/930741497 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9b3c0861b0 con 0x7f9b54150e60 2024-05-04T12:19:23.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.493+0000 7f9b53fff640 1 --2- 172.21.15.52:0/930741497 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f9b340678f0 0x7f9b34069db0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c007860 tx=0x7f9b4c0077f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:23.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.497+0000 7f9b50ff9640 1 -- 172.21.15.52:0/930741497 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9b3c012070 con 0x7f9b54150e60 2024-05-04T12:19:23.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:23.697+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/930741497 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9b5415b3d0 con 0x7f9b54150e60 2024-05-04T12:19:24.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.681+0000 7f9b50ff9640 1 -- 172.21.15.52:0/930741497 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f9b3c056db0 con 0x7f9b54150e60 2024-05-04T12:19:24.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.685+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/930741497 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f9b340678f0 msgr2=0x7f9b34069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:24.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.685+0000 7f9b5a9f8640 1 --2- 172.21.15.52:0/930741497 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f9b340678f0 0x7f9b34069db0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c007860 tx=0x7f9b4c0077f0 comp rx=0 tx=0).stop 2024-05-04T12:19:24.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.685+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/930741497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b54150e60 msgr2=0x7f9b54145d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:24.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.685+0000 7f9b5a9f8640 1 --2- 172.21.15.52:0/930741497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b54150e60 0x7f9b54145d70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00f350 tx=0x7f9b3c00f810 comp rx=0 tx=0).stop 2024-05-04T12:19:24.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.685+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/930741497 shutdown_connections 2024-05-04T12:19:24.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.685+0000 7f9b5a9f8640 1 --2- 172.21.15.52:0/930741497 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f9b340678f0 0x7f9b34069db0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:24.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.685+0000 7f9b5a9f8640 1 --2- 172.21.15.52:0/930741497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b5415e630 0x7f9b5415eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:24.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.685+0000 7f9b5a9f8640 1 --2- 172.21.15.52:0/930741497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b54150e60 0x7f9b54145d70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:24.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.685+0000 7f9b5a9f8640 1 --2- 172.21.15.52:0/930741497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b40004d50 0x7f9b540cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:24.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.685+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/930741497 >> 172.21.15.52:0/930741497 conn(0x7f9b540d12c0 msgr2=0x7f9b5415b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:24.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.685+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/930741497 shutdown_connections 2024-05-04T12:19:24.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.685+0000 7f9b5a9f8640 1 -- 172.21.15.52:0/930741497 wait complete. 2024-05-04T12:19:24.708 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-04T12:19:24.708 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-04T12:19:24.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.833+0000 7feae85cd640 1 Processor -- start 2024-05-04T12:19:24.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.833+0000 7feae85cd640 1 -- start start 2024-05-04T12:19:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae85cd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feae0144570 0x7feae0143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae85cd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feae0143e80 0x7feae01418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae85cd640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feae00d2710 con 0x7feae0141010 2024-05-04T12:19:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae85cd640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7feae00d29c0 con 0x7feae0143e80 2024-05-04T12:19:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae85cd640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7feae00d2ca0 con 0x7feae0144570 2024-05-04T12:19:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae6b6a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feae0143e80 0x7feae01418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae6b6a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feae0143e80 0x7feae01418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43856/0 (socket says 172.21.15.52:43856) 2024-05-04T12:19:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae6b6a640 1 -- 172.21.15.52:0/2316959741 learned_addr learned my addr 172.21.15.52:0/2316959741 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae5b68640 1 --2- 172.21.15.52:0/2316959741 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feae0144570 0x7feae0143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae6b6a640 1 -- 172.21.15.52:0/2316959741 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feae0144570 msgr2=0x7feae0143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:24.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae6b6a640 1 --2- 172.21.15.52:0/2316959741 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feae0144570 0x7feae0143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:24.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae6b6a640 1 -- 172.21.15.52:0/2316959741 >> v1:172.21.15.111:6789/0 conn(0x7feae0141010 legacy=0x7feae00d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:24.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae6b6a640 1 -- 172.21.15.52:0/2316959741 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae00d2f80 con 0x7feae0143e80 2024-05-04T12:19:24.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae5b68640 1 --2- 172.21.15.52:0/2316959741 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feae0144570 0x7feae0143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:24.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae6b6a640 1 --2- 172.21.15.52:0/2316959741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feae0143e80 0x7feae01418b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fead8009e50 tx=0x7fead801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5477323e692bdece server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:24.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae5367640 1 -- 172.21.15.52:0/2316959741 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead80a56d0 con 0x7feae0143e80 2024-05-04T12:19:24.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae5367640 1 -- 172.21.15.52:0/2316959741 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fead80a5870 con 0x7feae0143e80 2024-05-04T12:19:24.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.837+0000 7feae5367640 1 -- 172.21.15.52:0/2316959741 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead80a5c40 con 0x7feae0143e80 2024-05-04T12:19:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 -- 172.21.15.52:0/2316959741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feae0143e80 msgr2=0x7feae01418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 --2- 172.21.15.52:0/2316959741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feae0143e80 0x7feae01418b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fead8009e50 tx=0x7fead801c960 comp rx=0 tx=0).stop 2024-05-04T12:19:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 -- 172.21.15.52:0/2316959741 shutdown_connections 2024-05-04T12:19:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 --2- 172.21.15.52:0/2316959741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feae0143e80 0x7feae01418b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 --2- 172.21.15.52:0/2316959741 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feae0144570 0x7feae0143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 -- 172.21.15.52:0/2316959741 >> 172.21.15.52:0/2316959741 conn(0x7feae0072be0 msgr2=0x7feae00d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:24.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 -- 172.21.15.52:0/2316959741 shutdown_connections 2024-05-04T12:19:24.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 -- 172.21.15.52:0/2316959741 wait complete. 2024-05-04T12:19:24.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 Processor -- start 2024-05-04T12:19:24.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 -- start start 2024-05-04T12:19:24.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feae0141010 0x7feae00d0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:24.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feae0143e80 0x7feae00ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:24.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feae0144570 0x7feae0157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:24.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feae0142780 con 0x7feae0141010 2024-05-04T12:19:24.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7feae0142600 con 0x7feae0143e80 2024-05-04T12:19:24.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7feae0142900 con 0x7feae0144570 2024-05-04T12:19:24.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae6369640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feae0141010 0x7feae00d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:24.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae6369640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feae0141010 0x7feae00d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43860/0 (socket says 172.21.15.52:43860) 2024-05-04T12:19:24.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae6369640 1 -- 172.21.15.52:0/2362264439 learned_addr learned my addr 172.21.15.52:0/2362264439 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:24.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae5b68640 1 --2- 172.21.15.52:0/2362264439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feae0143e80 0x7feae00ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:24.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae6b6a640 1 --2- 172.21.15.52:0/2362264439 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feae0144570 0x7feae0157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:24.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae6369640 1 -- 172.21.15.52:0/2362264439 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feae0144570 msgr2=0x7feae0157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:24.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae6369640 1 --2- 172.21.15.52:0/2362264439 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feae0144570 0x7feae0157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:24.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae6369640 1 -- 172.21.15.52:0/2362264439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feae0143e80 msgr2=0x7feae00ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:24.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae6369640 1 --2- 172.21.15.52:0/2362264439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feae0143e80 0x7feae00ace80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:24.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae6369640 1 -- 172.21.15.52:0/2362264439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae0158740 con 0x7feae0141010 2024-05-04T12:19:24.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae6369640 1 --2- 172.21.15.52:0/2362264439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feae0141010 0x7feae00d0640 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7feadc00b410 tx=0x7feadc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:24.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7fead6ffd640 1 -- 172.21.15.52:0/2362264439 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc015030 con 0x7feae0141010 2024-05-04T12:19:24.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7fead6ffd640 1 -- 172.21.15.52:0/2362264439 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feadc004020 con 0x7feae0141010 2024-05-04T12:19:24.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 -- 172.21.15.52:0/2362264439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae0158180 con 0x7feae0141010 2024-05-04T12:19:24.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7fead6ffd640 1 -- 172.21.15.52:0/2362264439 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc004450 con 0x7feae0141010 2024-05-04T12:19:24.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.841+0000 7feae85cd640 1 -- 172.21.15.52:0/2362264439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae00cf180 con 0x7feae0141010 2024-05-04T12:19:24.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.845+0000 7feae85cd640 1 -- 172.21.15.52:0/2362264439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae0062e50 con 0x7feae0141010 2024-05-04T12:19:24.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.845+0000 7fead6ffd640 1 -- 172.21.15.52:0/2362264439 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feadc018070 con 0x7feae0141010 2024-05-04T12:19:24.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.845+0000 7fead6ffd640 1 --2- 172.21.15.52:0/2362264439 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7feac40678f0 0x7feac4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:24.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.845+0000 7fead6ffd640 1 -- 172.21.15.52:0/2362264439 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7feadc085880 con 0x7feae0141010 2024-05-04T12:19:24.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.845+0000 7feae5b68640 1 --2- 172.21.15.52:0/2362264439 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7feac40678f0 0x7feac4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:24.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.845+0000 7feae5b68640 1 --2- 172.21.15.52:0/2362264439 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7feac40678f0 0x7feac4069db0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fead0006fd0 tx=0x7fead0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:24.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:24.849+0000 7fead6ffd640 1 -- 172.21.15.52:0/2362264439 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7feadc0563c0 con 0x7feae0141010 2024-05-04T12:19:25.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.045+0000 7feae85cd640 1 -- 172.21.15.52:0/2362264439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7feae0075320 con 0x7feae0141010 2024-05-04T12:19:25.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.693+0000 7fead6ffd640 1 -- 172.21.15.52:0/2362264439 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7feadc055cc0 con 0x7feae0141010 2024-05-04T12:19:25.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.697+0000 7feae85cd640 1 -- 172.21.15.52:0/2362264439 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7feac40678f0 msgr2=0x7feac4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:25.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.697+0000 7feae85cd640 1 --2- 172.21.15.52:0/2362264439 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7feac40678f0 0x7feac4069db0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fead0006fd0 tx=0x7fead0008040 comp rx=0 tx=0).stop 2024-05-04T12:19:25.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.697+0000 7feae85cd640 1 -- 172.21.15.52:0/2362264439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feae0141010 msgr2=0x7feae00d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:25.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.697+0000 7feae85cd640 1 --2- 172.21.15.52:0/2362264439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feae0141010 0x7feae00d0640 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7feadc00b410 tx=0x7feadc00b8d0 comp rx=0 tx=0).stop 2024-05-04T12:19:25.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.697+0000 7feae85cd640 1 -- 172.21.15.52:0/2362264439 shutdown_connections 2024-05-04T12:19:25.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.697+0000 7feae85cd640 1 --2- 172.21.15.52:0/2362264439 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7feac40678f0 0x7feac4069db0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:25.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.697+0000 7feae85cd640 1 --2- 172.21.15.52:0/2362264439 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feae0144570 0x7feae0157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:25.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.697+0000 7feae85cd640 1 --2- 172.21.15.52:0/2362264439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feae0143e80 0x7feae00ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:25.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.697+0000 7feae85cd640 1 --2- 172.21.15.52:0/2362264439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feae0141010 0x7feae00d0640 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:25.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.697+0000 7feae85cd640 1 -- 172.21.15.52:0/2362264439 >> 172.21.15.52:0/2362264439 conn(0x7feae0072be0 msgr2=0x7feae015c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:25.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.697+0000 7feae85cd640 1 -- 172.21.15.52:0/2362264439 shutdown_connections 2024-05-04T12:19:25.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.697+0000 7feae85cd640 1 -- 172.21.15.52:0/2362264439 wait complete. 2024-05-04T12:19:25.725 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-04T12:19:25.725 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T12:19:25.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.849+0000 7eff2fd6b640 1 Processor -- start 2024-05-04T12:19:25.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.849+0000 7eff2fd6b640 1 -- start start 2024-05-04T12:19:25.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2fd6b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7eff280aa0c0 0x7eff280ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:25.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2fd6b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7eff280aa550 0x7eff280cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:25.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2fd6b640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff2806d1b0 con 0x7eff2815bf50 2024-05-04T12:19:25.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2fd6b640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7eff2815ca40 con 0x7eff280aa0c0 2024-05-04T12:19:25.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2fd6b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7eff2806d490 con 0x7eff280aa550 2024-05-04T12:19:25.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2d306640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7eff280aa0c0 0x7eff280ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:25.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2db07640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7eff2815bf50 0x7eff2815c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:37806/0 (socket says 172.21.15.52:37806) 2024-05-04T12:19:25.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2db07640 1 -- 172.21.15.52:0/888224604 learned_addr learned my addr 172.21.15.52:0/888224604 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:25.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2e308640 1 --2- 172.21.15.52:0/888224604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7eff280aa550 0x7eff280cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:25.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2d306640 1 -- 172.21.15.52:0/888224604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7eff280aa550 msgr2=0x7eff280cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:25.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2d306640 1 --2- 172.21.15.52:0/888224604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7eff280aa550 0x7eff280cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:25.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2d306640 1 -- 172.21.15.52:0/888224604 >> v1:172.21.15.52:6789/0 conn(0x7eff2815bf50 legacy=0x7eff2815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:25.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2cb05640 1 -- 172.21.15.52:0/888224604 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 667704663 0 0) 0x7eff2806d1b0 con 0x7eff2815bf50 2024-05-04T12:19:25.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2d306640 1 -- 172.21.15.52:0/888224604 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2806d770 con 0x7eff280aa0c0 2024-05-04T12:19:25.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2e308640 1 --2- 172.21.15.52:0/888224604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7eff280aa550 0x7eff280cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:25.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2d306640 1 --2- 172.21.15.52:0/888224604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff280aa0c0 0x7eff280ccb50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7eff180097c0 tx=0x7eff1801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b02982b229fa307 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:25.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2cb05640 1 -- 172.21.15.52:0/888224604 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff180aa5f0 con 0x7eff280aa0c0 2024-05-04T12:19:25.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2cb05640 1 -- 172.21.15.52:0/888224604 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff180af070 con 0x7eff280aa0c0 2024-05-04T12:19:25.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.853+0000 7eff2cb05640 1 -- 172.21.15.52:0/888224604 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff180aa9b0 con 0x7eff280aa0c0 2024-05-04T12:19:25.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 -- 172.21.15.52:0/888224604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff280aa0c0 msgr2=0x7eff280ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:25.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 --2- 172.21.15.52:0/888224604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff280aa0c0 0x7eff280ccb50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7eff180097c0 tx=0x7eff1801c470 comp rx=0 tx=0).stop 2024-05-04T12:19:25.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 -- 172.21.15.52:0/888224604 shutdown_connections 2024-05-04T12:19:25.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 --2- 172.21.15.52:0/888224604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7eff280aa550 0x7eff280cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:25.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 --2- 172.21.15.52:0/888224604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff280aa0c0 0x7eff280ccb50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:25.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 -- 172.21.15.52:0/888224604 >> 172.21.15.52:0/888224604 conn(0x7eff280b14a0 msgr2=0x7eff280b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:25.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 -- 172.21.15.52:0/888224604 shutdown_connections 2024-05-04T12:19:25.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 -- 172.21.15.52:0/888224604 wait complete. 2024-05-04T12:19:25.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 Processor -- start 2024-05-04T12:19:25.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 -- start start 2024-05-04T12:19:25.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff280aa0c0 0x7eff2814f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff280aa550 0x7eff28149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7eff2815bf50 0x7eff28151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff2806ebe0 con 0x7eff280aa0c0 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7eff2806ea60 con 0x7eff280aa550 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7eff2806ed60 con 0x7eff2815bf50 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2db07640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff280aa0c0 0x7eff2814f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2db07640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff280aa0c0 0x7eff2814f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43882/0 (socket says 172.21.15.52:43882) 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2db07640 1 -- 172.21.15.52:0/2022566151 learned_addr learned my addr 172.21.15.52:0/2022566151 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2e308640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7eff2815bf50 0x7eff28151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2d306640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff280aa550 0x7eff28149a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2db07640 1 -- 172.21.15.52:0/2022566151 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7eff2815bf50 msgr2=0x7eff28151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2db07640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7eff2815bf50 0x7eff28151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2db07640 1 -- 172.21.15.52:0/2022566151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff280aa550 msgr2=0x7eff28149a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2db07640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff280aa550 0x7eff28149a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:25.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2db07640 1 -- 172.21.15.52:0/2022566151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff28152480 con 0x7eff280aa0c0 2024-05-04T12:19:25.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2e308640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7eff2815bf50 0x7eff28151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:19:25.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2db07640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff280aa0c0 0x7eff2814f700 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7eff2400c480 tx=0x7eff2400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:25.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff1e7fc640 1 -- 172.21.15.52:0/2022566151 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24018070 con 0x7eff280aa0c0 2024-05-04T12:19:25.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff1e7fc640 1 -- 172.21.15.52:0/2022566151 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff24007500 con 0x7eff280aa0c0 2024-05-04T12:19:25.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff1e7fc640 1 -- 172.21.15.52:0/2022566151 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24004a30 con 0x7eff280aa0c0 2024-05-04T12:19:25.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 -- 172.21.15.52:0/2022566151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff28152760 con 0x7eff280aa0c0 2024-05-04T12:19:25.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.857+0000 7eff2fd6b640 1 -- 172.21.15.52:0/2022566151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff28152dd0 con 0x7eff280aa0c0 2024-05-04T12:19:25.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.861+0000 7eff2fd6b640 1 -- 172.21.15.52:0/2022566151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff28062e50 con 0x7eff280aa0c0 2024-05-04T12:19:25.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.861+0000 7eff1e7fc640 1 -- 172.21.15.52:0/2022566151 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7eff24004bd0 con 0x7eff280aa0c0 2024-05-04T12:19:25.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.861+0000 7eff1e7fc640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7eff0c0678f0 0x7eff0c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:25.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.861+0000 7eff1e7fc640 1 -- 172.21.15.52:0/2022566151 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7eff24084e90 con 0x7eff280aa0c0 2024-05-04T12:19:25.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.861+0000 7eff2d306640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7eff0c0678f0 0x7eff0c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:25.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.861+0000 7eff2d306640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7eff0c0678f0 0x7eff0c069db0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7eff180ae000 tx=0x7eff180a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:25.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:25.865+0000 7eff1e7fc640 1 -- 172.21.15.52:0/2022566151 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7eff24055a60 con 0x7eff280aa0c0 2024-05-04T12:19:26.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.057+0000 7eff2fd6b640 1 -- 172.21.15.52:0/2022566151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff28075320 con 0x7eff280aa0c0 2024-05-04T12:19:26.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.061+0000 7eff1e7fc640 1 -- 172.21.15.52:0/2022566151 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5184 (secure 0 0 0) 0x7eff24055360 con 0x7eff280aa0c0 2024-05-04T12:19:26.063 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:26.063 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2438654614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2438654614},{"type":"v1","addr":"172.21.15.111:6835","nonce":2438654614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/796226117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":796226117},{"type":"v1","addr":"172.21.15.52:6835","nonce":796226117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3866938727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3866938727},{"type":"v1","addr":"172.21.15.111:6833","nonce":3866938727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-04T12:19:22.017880+0000","modified":"2024-05-04T12:19:25.052316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5199":{"gid":5199,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6837/1620779861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1620779861},{"type":"v1","addr":"172.21.15.52:6837","nonce":1620779861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5199,"qdb_cluster":[5199]},"id":2}]} 2024-05-04T12:19:26.064 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 16 2024-05-04T12:19:26.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.061+0000 7eff2fd6b640 1 -- 172.21.15.52:0/2022566151 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7eff0c0678f0 msgr2=0x7eff0c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.061+0000 7eff2fd6b640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7eff0c0678f0 0x7eff0c069db0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7eff180ae000 tx=0x7eff180a7000 comp rx=0 tx=0).stop 2024-05-04T12:19:26.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.061+0000 7eff2fd6b640 1 -- 172.21.15.52:0/2022566151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff280aa0c0 msgr2=0x7eff2814f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.061+0000 7eff2fd6b640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff280aa0c0 0x7eff2814f700 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7eff2400c480 tx=0x7eff2400c940 comp rx=0 tx=0).stop 2024-05-04T12:19:26.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.061+0000 7eff2fd6b640 1 -- 172.21.15.52:0/2022566151 shutdown_connections 2024-05-04T12:19:26.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.061+0000 7eff2fd6b640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7eff0c0678f0 0x7eff0c069db0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.061+0000 7eff2fd6b640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7eff2815bf50 0x7eff28151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.061+0000 7eff2fd6b640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff280aa550 0x7eff28149a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.061+0000 7eff2fd6b640 1 --2- 172.21.15.52:0/2022566151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff280aa0c0 0x7eff2814f700 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.061+0000 7eff2fd6b640 1 -- 172.21.15.52:0/2022566151 >> 172.21.15.52:0/2022566151 conn(0x7eff280b14a0 msgr2=0x7eff280cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:26.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.061+0000 7eff2fd6b640 1 -- 172.21.15.52:0/2022566151 shutdown_connections 2024-05-04T12:19:26.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.061+0000 7eff2fd6b640 1 -- 172.21.15.52:0/2022566151 wait complete. 2024-05-04T12:19:26.081 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T12:19:26.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c9e80640 1 Processor -- start 2024-05-04T12:19:26.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c9e80640 1 -- start start 2024-05-04T12:19:26.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c9e80640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f93c40cd260 0x7f93c406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c9e80640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93c406d1b0 con 0x7f93c415bf50 2024-05-04T12:19:26.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c9e80640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93c406d490 con 0x7f93c40aa0c0 2024-05-04T12:19:26.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c9e80640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f93c415ca40 con 0x7f93c40cd260 2024-05-04T12:19:26.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c37fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f93c415bf50 0x7f93c415c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:41090/0 (socket says 172.21.15.52:41090) 2024-05-04T12:19:26.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c37fe640 1 -- 172.21.15.52:0/1214887378 learned_addr learned my addr 172.21.15.52:0/1214887378 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:26.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c3fff640 1 --2- 172.21.15.52:0/1214887378 >> v2:172.21.15.111:3300/0 conn(0x7f93c40cd260 0x7f93c406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c27fc640 1 -- 172.21.15.52:0/1214887378 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2836320548 0 0) 0x7f93c406d1b0 con 0x7f93c415bf50 2024-05-04T12:19:26.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c27fc640 1 -- 172.21.15.52:0/1214887378 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93ac003530 con 0x7f93c415bf50 2024-05-04T12:19:26.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c27fc640 1 -- 172.21.15.52:0/1214887378 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1681321122 0 0) 0x7f93c406d490 con 0x7f93c40aa0c0 2024-05-04T12:19:26.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c27fc640 1 -- 172.21.15.52:0/1214887378 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93c406d1b0 con 0x7f93c40aa0c0 2024-05-04T12:19:26.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c27fc640 1 -- 172.21.15.52:0/1214887378 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 668113786 0 0) 0x7f93ac003530 con 0x7f93c415bf50 2024-05-04T12:19:26.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c3fff640 1 -- 172.21.15.52:0/1214887378 >> v1:172.21.15.111:6789/0 conn(0x7f93c40aa0c0 legacy=0x7f93c40ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c3fff640 1 -- 172.21.15.52:0/1214887378 >> v1:172.21.15.52:6789/0 conn(0x7f93c415bf50 legacy=0x7f93c415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.201+0000 7f93c3fff640 1 -- 172.21.15.52:0/1214887378 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c406d770 con 0x7f93c40cd260 2024-05-04T12:19:26.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.205+0000 7f93c3fff640 1 --2- 172.21.15.52:0/1214887378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c40cd260 0x7f93c406e250 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009d90 tx=0x7f93bc01c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fea7d1f17d16720 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:26.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.205+0000 7f93c27fc640 1 -- 172.21.15.52:0/1214887378 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0b2070 con 0x7f93c40cd260 2024-05-04T12:19:26.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.205+0000 7f93c27fc640 1 -- 172.21.15.52:0/1214887378 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93bc0a55b0 con 0x7f93c40cd260 2024-05-04T12:19:26.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.205+0000 7f93c27fc640 1 -- 172.21.15.52:0/1214887378 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4001680 con 0x7f93c40cd260 2024-05-04T12:19:26.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.205+0000 7f93c9e80640 1 -- 172.21.15.52:0/1214887378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c40cd260 msgr2=0x7f93c406e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.205+0000 7f93c9e80640 1 --2- 172.21.15.52:0/1214887378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c40cd260 0x7f93c406e250 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009d90 tx=0x7f93bc01c860 comp rx=0 tx=0).stop 2024-05-04T12:19:26.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.205+0000 7f93c9e80640 1 -- 172.21.15.52:0/1214887378 shutdown_connections 2024-05-04T12:19:26.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.205+0000 7f93c9e80640 1 --2- 172.21.15.52:0/1214887378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c40cd260 0x7f93c406e250 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.205+0000 7f93c9e80640 1 -- 172.21.15.52:0/1214887378 >> 172.21.15.52:0/1214887378 conn(0x7f93c40b14a0 msgr2=0x7f93c40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.205+0000 7f93c9e80640 1 -- 172.21.15.52:0/1214887378 shutdown_connections 2024-05-04T12:19:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.205+0000 7f93c9e80640 1 -- 172.21.15.52:0/1214887378 wait complete. 2024-05-04T12:19:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.205+0000 7f93c9e80640 1 Processor -- start 2024-05-04T12:19:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.205+0000 7f93c9e80640 1 -- start start 2024-05-04T12:19:26.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c9e80640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93c40aa0c0 0x7f93c41521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c9e80640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c40cd260 0x7f93c4154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c9e80640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93c415bf50 0x7f93c413d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c9e80640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f93c416a1c0 con 0x7f93c415bf50 2024-05-04T12:19:26.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c9e80640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f93c416a040 con 0x7f93c40cd260 2024-05-04T12:19:26.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c9e80640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f93c416a340 con 0x7f93c40aa0c0 2024-05-04T12:19:26.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93c415bf50 0x7f93c413d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93c415bf50 0x7f93c413d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59560/0 (socket says 172.21.15.52:59560) 2024-05-04T12:19:26.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c3fff640 1 -- 172.21.15.52:0/4141988735 learned_addr learned my addr 172.21.15.52:0/4141988735 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:26.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c37fe640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93c40aa0c0 0x7f93c41521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c2ffd640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c40cd260 0x7f93c4154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c3fff640 1 -- 172.21.15.52:0/4141988735 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93c40aa0c0 msgr2=0x7f93c41521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c3fff640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93c40aa0c0 0x7f93c41521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c3fff640 1 -- 172.21.15.52:0/4141988735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c40cd260 msgr2=0x7f93c4154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c3fff640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c40cd260 0x7f93c4154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c3fff640 1 -- 172.21.15.52:0/4141988735 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c4154960 con 0x7f93c415bf50 2024-05-04T12:19:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c37fe640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93c40aa0c0 0x7f93c41521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c3fff640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93c415bf50 0x7f93c413d2d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009d90 tx=0x7f93bc019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93abfff640 1 -- 172.21.15.52:0/4141988735 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0c0070 con 0x7f93c415bf50 2024-05-04T12:19:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93abfff640 1 -- 172.21.15.52:0/4141988735 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93bc0039a0 con 0x7f93c415bf50 2024-05-04T12:19:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c9e80640 1 -- 172.21.15.52:0/4141988735 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c4145120 con 0x7f93c415bf50 2024-05-04T12:19:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93abfff640 1 -- 172.21.15.52:0/4141988735 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0b2040 con 0x7f93c415bf50 2024-05-04T12:19:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c9e80640 1 -- 172.21.15.52:0/4141988735 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c41456e0 con 0x7f93c415bf50 2024-05-04T12:19:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c9e80640 1 -- 172.21.15.52:0/4141988735 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c4062e50 con 0x7f93c415bf50 2024-05-04T12:19:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93abfff640 1 -- 172.21.15.52:0/4141988735 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f93bc0c3070 con 0x7f93c415bf50 2024-05-04T12:19:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93abfff640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f93b00678c0 0x7f93b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93abfff640 1 -- 172.21.15.52:0/4141988735 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f93bc11fe20 con 0x7f93c415bf50 2024-05-04T12:19:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.209+0000 7f93c37fe640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f93b00678c0 0x7f93b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.213+0000 7f93c37fe640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f93b00678c0 0x7f93b0069d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f93b4000fc0 tx=0x7f93b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:26.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.213+0000 7f93abfff640 1 -- 172.21.15.52:0/4141988735 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f93bc016480 con 0x7f93c415bf50 2024-05-04T12:19:26.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.373+0000 7f93c9e80640 1 -- 172.21.15.52:0/4141988735 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93c4075320 con 0x7f93c415bf50 2024-05-04T12:19:26.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.373+0000 7f93abfff640 1 -- 172.21.15.52:0/4141988735 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16681 (secure 0 0 0) 0x7f93bc0f0a80 con 0x7f93c415bf50 2024-05-04T12:19:26.379 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:26.379 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":34,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:19:23.639019+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:19:20.589701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-04T12:19:20.997186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T12:19:26.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.377+0000 7f93c9e80640 1 -- 172.21.15.52:0/4141988735 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f93b00678c0 msgr2=0x7f93b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.377+0000 7f93c9e80640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f93b00678c0 0x7f93b0069d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f93b4000fc0 tx=0x7f93b4009040 comp rx=0 tx=0).stop 2024-05-04T12:19:26.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.377+0000 7f93c9e80640 1 -- 172.21.15.52:0/4141988735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93c415bf50 msgr2=0x7f93c413d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.377+0000 7f93c9e80640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93c415bf50 0x7f93c413d2d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009d90 tx=0x7f93bc019600 comp rx=0 tx=0).stop 2024-05-04T12:19:26.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.377+0000 7f93c9e80640 1 -- 172.21.15.52:0/4141988735 shutdown_connections 2024-05-04T12:19:26.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.377+0000 7f93c9e80640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f93b00678c0 0x7f93b0069d80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.377+0000 7f93c9e80640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93c415bf50 0x7f93c413d2d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.377+0000 7f93c9e80640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c40cd260 0x7f93c4154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.377+0000 7f93c9e80640 1 --2- 172.21.15.52:0/4141988735 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93c40aa0c0 0x7f93c41521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.377+0000 7f93c9e80640 1 -- 172.21.15.52:0/4141988735 >> 172.21.15.52:0/4141988735 conn(0x7f93c40b14a0 msgr2=0x7f93c4159550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:26.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.377+0000 7f93c9e80640 1 -- 172.21.15.52:0/4141988735 shutdown_connections 2024-05-04T12:19:26.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.377+0000 7f93c9e80640 1 -- 172.21.15.52:0/4141988735 wait complete. 2024-05-04T12:19:26.397 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-04T12:19:26.397 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-04T12:19:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.505+0000 7f193872b640 1 Processor -- start 2024-05-04T12:19:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.505+0000 7f193872b640 1 -- start start 2024-05-04T12:19:26.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f193872b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19300aa0c0 0x7f19300aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f193872b640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f193006d1b0 con 0x7f193015bf50 2024-05-04T12:19:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f193872b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f193006d490 con 0x7f19300ccb50 2024-05-04T12:19:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f193872b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f193015ca40 con 0x7f19300aa0c0 2024-05-04T12:19:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19364c7640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f193015bf50 0x7f193015c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:41098/0 (socket says 172.21.15.52:41098) 2024-05-04T12:19:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19364c7640 1 -- 172.21.15.52:0/3789887729 learned_addr learned my addr 172.21.15.52:0/3789887729 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 -- 172.21.15.52:0/3789887729 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1636844511 0 0) 0x7f193006d1b0 con 0x7f193015bf50 2024-05-04T12:19:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f1935cc6640 1 --2- 172.21.15.52:0/3789887729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19300aa0c0 0x7f19300aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 -- 172.21.15.52:0/3789887729 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f191c003530 con 0x7f193015bf50 2024-05-04T12:19:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 -- 172.21.15.52:0/3789887729 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 690662929 0 0) 0x7f191c003530 con 0x7f193015bf50 2024-05-04T12:19:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 -- 172.21.15.52:0/3789887729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19300aa0c0 msgr2=0x7f19300aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 --2- 172.21.15.52:0/3789887729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19300aa0c0 0x7f19300aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 -- 172.21.15.52:0/3789887729 >> v1:172.21.15.111:6789/0 conn(0x7f19300ccb50 legacy=0x7f193006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 -- 172.21.15.52:0/3789887729 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193006d770 con 0x7f193015bf50 2024-05-04T12:19:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 -- 172.21.15.52:0/3789887729 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f192c001680 con 0x7f193015bf50 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 -- 172.21.15.52:0/3789887729 >> v1:172.21.15.52:6789/0 conn(0x7f193015bf50 legacy=0x7f193015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 --2- 172.21.15.52:0/3789887729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f191c0039f0 0x7f191c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 --2- 172.21.15.52:0/3789887729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19300aa0c0 0x7f191c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 --2- 172.21.15.52:0/3789887729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f191c004d50 0x7f191c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 -- 172.21.15.52:0/3789887729 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f193015ca40 con 0x7f191c004d50 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 -- 172.21.15.52:0/3789887729 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f191c00ba80 con 0x7f191c0039f0 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19354c5640 1 -- 172.21.15.52:0/3789887729 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f191c00bc80 con 0x7f19300aa0c0 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f1936cc8640 1 --2- 172.21.15.52:0/3789887729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f191c004d50 0x7f191c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f19364c7640 1 --2- 172.21.15.52:0/3789887729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f191c0039f0 0x7f191c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f1935cc6640 1 --2- 172.21.15.52:0/3789887729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19300aa0c0 0x7f191c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f1936cc8640 1 -- 172.21.15.52:0/3789887729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19300aa0c0 msgr2=0x7f191c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f1936cc8640 1 --2- 172.21.15.52:0/3789887729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19300aa0c0 0x7f191c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f1936cc8640 1 -- 172.21.15.52:0/3789887729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f191c0039f0 msgr2=0x7f191c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f1936cc8640 1 --2- 172.21.15.52:0/3789887729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f191c0039f0 0x7f191c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.509+0000 7f1936cc8640 1 -- 172.21.15.52:0/3789887729 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c003530 con 0x7f191c004d50 2024-05-04T12:19:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f1936cc8640 1 --2- 172.21.15.52:0/3789887729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f191c004d50 0x7f191c0051d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1928001e40 tx=0x7f192801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451a9f809954fc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f19354c5640 1 -- 172.21.15.52:0/3789887729 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928001a90 con 0x7f191c004d50 2024-05-04T12:19:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f19354c5640 1 -- 172.21.15.52:0/3789887729 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1928003b80 con 0x7f191c004d50 2024-05-04T12:19:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f19354c5640 1 -- 172.21.15.52:0/3789887729 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928005080 con 0x7f191c004d50 2024-05-04T12:19:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 -- 172.21.15.52:0/3789887729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f191c004d50 msgr2=0x7f191c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 --2- 172.21.15.52:0/3789887729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f191c004d50 0x7f191c0051d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1928001e40 tx=0x7f192801c760 comp rx=0 tx=0).stop 2024-05-04T12:19:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f1936cc8640 1 -- 172.21.15.52:0/3789887729 reap_dead start 2024-05-04T12:19:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 -- 172.21.15.52:0/3789887729 shutdown_connections 2024-05-04T12:19:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 -- 172.21.15.52:0/3789887729 >> 172.21.15.52:0/3789887729 conn(0x7f19300b14a0 msgr2=0x7f19300b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 -- 172.21.15.52:0/3789887729 shutdown_connections 2024-05-04T12:19:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 -- 172.21.15.52:0/3789887729 wait complete. 2024-05-04T12:19:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 Processor -- start 2024-05-04T12:19:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 -- start start 2024-05-04T12:19:26.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f191c004d50 0x7f193014e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1930150f10 0x7f19301f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19301f1fc0 0x7f19300a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f193006cd60 con 0x7f191c004d50 2024-05-04T12:19:26.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f193006dd30 con 0x7f1930150f10 2024-05-04T12:19:26.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f193006cee0 con 0x7f19301f1fc0 2024-05-04T12:19:26.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f19364c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f191c004d50 0x7f193014e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f19364c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f191c004d50 0x7f193014e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59578/0 (socket says 172.21.15.52:59578) 2024-05-04T12:19:26.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f19364c7640 1 -- 172.21.15.52:0/1045360493 learned_addr learned my addr 172.21.15.52:0/1045360493 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:26.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f1935cc6640 1 --2- 172.21.15.52:0/1045360493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1930150f10 0x7f19301f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f1936cc8640 1 --2- 172.21.15.52:0/1045360493 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19301f1fc0 0x7f19300a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f19364c7640 1 -- 172.21.15.52:0/1045360493 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19301f1fc0 msgr2=0x7f19300a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f19364c7640 1 --2- 172.21.15.52:0/1045360493 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19301f1fc0 0x7f19300a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f19364c7640 1 -- 172.21.15.52:0/1045360493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1930150f10 msgr2=0x7f19301f1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f19364c7640 1 --2- 172.21.15.52:0/1045360493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1930150f10 0x7f19301f1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f19364c7640 1 -- 172.21.15.52:0/1045360493 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19301f2590 con 0x7f191c004d50 2024-05-04T12:19:26.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f19364c7640 1 --2- 172.21.15.52:0/1045360493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f191c004d50 0x7f193014e600 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f192c00e7e0 tx=0x7f192c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:26.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f1926ffd640 1 -- 172.21.15.52:0/1045360493 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c00c8d0 con 0x7f191c004d50 2024-05-04T12:19:26.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f1926ffd640 1 -- 172.21.15.52:0/1045360493 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f192c018070 con 0x7f191c004d50 2024-05-04T12:19:26.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 -- 172.21.15.52:0/1045360493 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19301f2870 con 0x7f191c004d50 2024-05-04T12:19:26.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f1926ffd640 1 -- 172.21.15.52:0/1045360493 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c00ccd0 con 0x7f191c004d50 2024-05-04T12:19:26.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.513+0000 7f193872b640 1 -- 172.21.15.52:0/1045360493 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19301509f0 con 0x7f191c004d50 2024-05-04T12:19:26.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.517+0000 7f193872b640 1 -- 172.21.15.52:0/1045360493 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1930150bf0 con 0x7f191c004d50 2024-05-04T12:19:26.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.517+0000 7f1926ffd640 1 -- 172.21.15.52:0/1045360493 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f192c026430 con 0x7f191c004d50 2024-05-04T12:19:26.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.517+0000 7f1926ffd640 1 --2- 172.21.15.52:0/1045360493 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19140678f0 0x7f1914069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.517+0000 7f1926ffd640 1 -- 172.21.15.52:0/1045360493 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f192c086360 con 0x7f191c004d50 2024-05-04T12:19:26.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.517+0000 7f1935cc6640 1 --2- 172.21.15.52:0/1045360493 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19140678f0 0x7f1914069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.517+0000 7f1935cc6640 1 --2- 172.21.15.52:0/1045360493 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19140678f0 0x7f1914069db0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f19300cf450 tx=0x7f1920008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:26.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.521+0000 7f1926ffd640 1 -- 172.21.15.52:0/1045360493 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1930150bf0 con 0x7f191c004d50 2024-05-04T12:19:26.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.677+0000 7f193872b640 1 -- 172.21.15.52:0/1045360493 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1930150500 con 0x7f19140678f0 2024-05-04T12:19:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.693+0000 7f1926ffd640 1 -- 172.21.15.52:0/1045360493 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237199 (secure 0 0 0) 0x7f1930150500 con 0x7f19140678f0 2024-05-04T12:19:26.700 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-04T12:19:26.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.697+0000 7f193872b640 1 -- 172.21.15.52:0/1045360493 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19140678f0 msgr2=0x7f1914069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.697+0000 7f193872b640 1 --2- 172.21.15.52:0/1045360493 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19140678f0 0x7f1914069db0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f19300cf450 tx=0x7f1920008040 comp rx=0 tx=0).stop 2024-05-04T12:19:26.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.697+0000 7f193872b640 1 -- 172.21.15.52:0/1045360493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f191c004d50 msgr2=0x7f193014e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.697+0000 7f193872b640 1 --2- 172.21.15.52:0/1045360493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f191c004d50 0x7f193014e600 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f192c00e7e0 tx=0x7f192c00eca0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.697+0000 7f193872b640 1 -- 172.21.15.52:0/1045360493 shutdown_connections 2024-05-04T12:19:26.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.697+0000 7f193872b640 1 --2- 172.21.15.52:0/1045360493 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f19140678f0 0x7f1914069db0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.697+0000 7f193872b640 1 --2- 172.21.15.52:0/1045360493 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19301f1fc0 0x7f19300a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.697+0000 7f193872b640 1 --2- 172.21.15.52:0/1045360493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1930150f10 0x7f19301f1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.697+0000 7f193872b640 1 --2- 172.21.15.52:0/1045360493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f191c004d50 0x7f193014e600 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.697+0000 7f193872b640 1 -- 172.21.15.52:0/1045360493 >> 172.21.15.52:0/1045360493 conn(0x7f19300b14a0 msgr2=0x7f193015c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:26.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.697+0000 7f193872b640 1 -- 172.21.15.52:0/1045360493 shutdown_connections 2024-05-04T12:19:26.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.697+0000 7f193872b640 1 -- 172.21.15.52:0/1045360493 wait complete. 2024-05-04T12:19:26.717 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-05-04T12:19:24.757930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236640,"kb_used_data":3784,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738080,"statfs":{"total":773094113280,"available":772851793920,"internally_reserved":0,"allocated":3874816,"data_stored":2191056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.132453"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:21.625858+0000","last_change":"2024-05-04T12:19:21.625858+0000","last_active":"2024-05-04T12:19:21.625858+0000","last_peered":"2024-05-04T12:19:21.625858+0000","last_clean":"2024-05-04T12:19:21.625858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:21.625858+0000","last_undegraded":"2024-05-04T12:19:21.625858+0000","last_fullsized":"2024-05-04T12:19:21.625858+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-04T12:19:20.617457+0000","last_change":"2024-05-04T12:19:20.617457+0000","last_active":"2024-05-04T12:19:20.617457+0000","last_peered":"2024-05-04T12:19:20.617457+0000","last_clean":"2024-05-04T12:19:20.617457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-04T12:19:20.617457+0000","last_undegraded":"2024-05-04T12:19:20.617457+0000","last_fullsized":"2024-05-04T12:19:20.617457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":186,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-04T12:19:16.707645+0000","last_change":"2024-05-04T12:18:12.644577+0000","last_active":"2024-05-04T12:19:16.707645+0000","last_peered":"2024-05-04T12:19:16.707645+0000","last_clean":"2024-05-04T12:19:16.707645+0000","last_became_active":"2024-05-04T12:18:12.640041+0000","last_became_peered":"2024-05-04T12:18:12.640041+0000","last_unstale":"2024-05-04T12:19:16.707645+0000","last_undegraded":"2024-05-04T12:19:16.707645+0000","last_fullsized":"2024-05-04T12:19:16.707645+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_clean_scrub_stamp":"2024-05-04T12:18:11.260157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:15:02.089387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":929792,"data_stored":716658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":126290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27364,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":335872,"data_stored":126290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27368,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":335872,"data_stored":126290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":126290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":126290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":126290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":929792,"data_stored":716658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T12:19:26.721 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-04T12:19:26.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.877+0000 7fba81b58640 1 Processor -- start 2024-05-04T12:19:26.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba81b58640 1 -- start start 2024-05-04T12:19:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba81b58640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fba7c15c7f0 0x7fba7c0723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba81b58640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba7c0cd190 0x7fba7c15c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba81b58640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba7c072aa0 con 0x7fba7c0ccdb0 2024-05-04T12:19:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba81b58640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fba7c070690 con 0x7fba7c15c7f0 2024-05-04T12:19:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba81b58640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c0708e0 con 0x7fba7c0cd190 2024-05-04T12:19:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7b7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fba7c0ccdb0 0x7fba7c158640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:41108/0 (socket says 172.21.15.52:41108) 2024-05-04T12:19:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7b7fe640 1 -- 172.21.15.52:0/1761405630 learned_addr learned my addr 172.21.15.52:0/1761405630 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba73fff640 1 --2- 172.21.15.52:0/1761405630 >> v2:172.21.15.111:3300/0 conn(0x7fba7c15c7f0 0x7fba7c0723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7bfff640 1 --2- 172.21.15.52:0/1761405630 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba7c0cd190 0x7fba7c15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 -- 172.21.15.52:0/1761405630 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2740588471 0 0) 0x7fba7c072aa0 con 0x7fba7c0ccdb0 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 -- 172.21.15.52:0/1761405630 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba68003530 con 0x7fba7c0ccdb0 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 -- 172.21.15.52:0/1761405630 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2218441071 0 0) 0x7fba68003530 con 0x7fba7c0ccdb0 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 -- 172.21.15.52:0/1761405630 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba7c0cd190 msgr2=0x7fba7c15c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 --2- 172.21.15.52:0/1761405630 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba7c0cd190 0x7fba7c15c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 -- 172.21.15.52:0/1761405630 >> v2:172.21.15.111:3300/0 conn(0x7fba7c15c7f0 msgr2=0x7fba7c0723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 --2- 172.21.15.52:0/1761405630 >> v2:172.21.15.111:3300/0 conn(0x7fba7c15c7f0 0x7fba7c0723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 -- 172.21.15.52:0/1761405630 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c070bc0 con 0x7fba7c0ccdb0 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 -- 172.21.15.52:0/1761405630 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7fba6c001680 con 0x7fba7c0ccdb0 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 -- 172.21.15.52:0/1761405630 >> v1:172.21.15.52:6789/0 conn(0x7fba7c0ccdb0 legacy=0x7fba7c158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 --2- 172.21.15.52:0/1761405630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba680039c0 0x7fba68003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 --2- 172.21.15.52:0/1761405630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba68004810 0x7fba68004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 --2- 172.21.15.52:0/1761405630 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba7c0cd190 0x7fba680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 -- 172.21.15.52:0/1761405630 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c070690 con 0x7fba680039c0 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 -- 172.21.15.52:0/1761405630 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c0708e0 con 0x7fba68004810 2024-05-04T12:19:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7affd640 1 -- 172.21.15.52:0/1761405630 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fba6800b9d0 con 0x7fba7c0cd190 2024-05-04T12:19:26.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7b7fe640 1 --2- 172.21.15.52:0/1761405630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba680039c0 0x7fba68003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba73fff640 1 --2- 172.21.15.52:0/1761405630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba68004810 0x7fba68004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7b7fe640 1 -- 172.21.15.52:0/1761405630 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba7c0cd190 msgr2=0x7fba680051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:26.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7b7fe640 1 --2- 172.21.15.52:0/1761405630 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba7c0cd190 0x7fba680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7b7fe640 1 -- 172.21.15.52:0/1761405630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba68004810 msgr2=0x7fba68004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:26.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7b7fe640 1 --2- 172.21.15.52:0/1761405630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba68004810 0x7fba68004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.881+0000 7fba7b7fe640 1 -- 172.21.15.52:0/1761405630 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba68003530 con 0x7fba680039c0 2024-05-04T12:19:26.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba7b7fe640 1 --2- 172.21.15.52:0/1761405630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba680039c0 0x7fba68003de0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00c270 tx=0x7fba6c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc48751c00e30302 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:26.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba7affd640 1 -- 172.21.15.52:0/1761405630 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c007d40 con 0x7fba680039c0 2024-05-04T12:19:26.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba7affd640 1 -- 172.21.15.52:0/1761405630 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c0a9020 con 0x7fba680039c0 2024-05-04T12:19:26.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba7affd640 1 -- 172.21.15.52:0/1761405630 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c005ae0 con 0x7fba680039c0 2024-05-04T12:19:26.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba81b58640 1 -- 172.21.15.52:0/1761405630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba680039c0 msgr2=0x7fba68003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba81b58640 1 --2- 172.21.15.52:0/1761405630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba680039c0 0x7fba68003de0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00c270 tx=0x7fba6c01cca0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba7bfff640 1 -- 172.21.15.52:0/1761405630 reap_dead start 2024-05-04T12:19:26.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba81b58640 1 -- 172.21.15.52:0/1761405630 shutdown_connections 2024-05-04T12:19:26.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba81b58640 1 -- 172.21.15.52:0/1761405630 >> 172.21.15.52:0/1761405630 conn(0x7fba7c0b14a0 msgr2=0x7fba7c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:26.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba81b58640 1 -- 172.21.15.52:0/1761405630 shutdown_connections 2024-05-04T12:19:26.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba81b58640 1 -- 172.21.15.52:0/1761405630 wait complete. 2024-05-04T12:19:26.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba81b58640 1 Processor -- start 2024-05-04T12:19:26.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba81b58640 1 -- start start 2024-05-04T12:19:26.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba81b58640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba680039c0 0x7fba7c14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba81b58640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba7c1611b0 0x7fba7c153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba81b58640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba7c14b200 0x7fba7c150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba81b58640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c06ea50 con 0x7fba7c1611b0 2024-05-04T12:19:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba81b58640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c06e8d0 con 0x7fba680039c0 2024-05-04T12:19:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba81b58640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c06ebd0 con 0x7fba7c14b200 2024-05-04T12:19:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba73fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba7c1611b0 0x7fba7c153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba73fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba7c1611b0 0x7fba7c153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59582/0 (socket says 172.21.15.52:59582) 2024-05-04T12:19:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.885+0000 7fba73fff640 1 -- 172.21.15.52:0/2182363650 learned_addr learned my addr 172.21.15.52:0/2182363650 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba7b7fe640 1 --2- 172.21.15.52:0/2182363650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba680039c0 0x7fba7c14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba7bfff640 1 --2- 172.21.15.52:0/2182363650 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba7c14b200 0x7fba7c150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba73fff640 1 -- 172.21.15.52:0/2182363650 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba7c14b200 msgr2=0x7fba7c150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba73fff640 1 --2- 172.21.15.52:0/2182363650 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba7c14b200 0x7fba7c150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba73fff640 1 -- 172.21.15.52:0/2182363650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba680039c0 msgr2=0x7fba7c14fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba73fff640 1 --2- 172.21.15.52:0/2182363650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba680039c0 0x7fba7c14fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba73fff640 1 -- 172.21.15.52:0/2182363650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c161630 con 0x7fba7c1611b0 2024-05-04T12:19:26.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba73fff640 1 --2- 172.21.15.52:0/2182363650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba7c1611b0 0x7fba7c153550 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fba640108f0 tx=0x7fba64010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:26.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba78ff9640 1 -- 172.21.15.52:0/2182363650 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6400f040 con 0x7fba7c1611b0 2024-05-04T12:19:26.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba78ff9640 1 -- 172.21.15.52:0/2182363650 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6400eb00 con 0x7fba7c1611b0 2024-05-04T12:19:26.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba78ff9640 1 -- 172.21.15.52:0/2182363650 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6401c400 con 0x7fba7c1611b0 2024-05-04T12:19:26.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba81b58640 1 -- 172.21.15.52:0/2182363650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c1501e0 con 0x7fba7c1611b0 2024-05-04T12:19:26.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba81b58640 1 -- 172.21.15.52:0/2182363650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c1540a0 con 0x7fba7c1611b0 2024-05-04T12:19:26.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba81b58640 1 -- 172.21.15.52:0/2182363650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7c062e50 con 0x7fba7c1611b0 2024-05-04T12:19:26.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba78ff9640 1 -- 172.21.15.52:0/2182363650 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fba6401c5a0 con 0x7fba7c1611b0 2024-05-04T12:19:26.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba78ff9640 1 --2- 172.21.15.52:0/2182363650 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fba5c0678c0 0x7fba5c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:26.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba78ff9640 1 -- 172.21.15.52:0/2182363650 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fba640861f0 con 0x7fba7c1611b0 2024-05-04T12:19:26.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba7b7fe640 1 --2- 172.21.15.52:0/2182363650 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fba5c0678c0 0x7fba5c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:26.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.889+0000 7fba7b7fe640 1 --2- 172.21.15.52:0/2182363650 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fba5c0678c0 0x7fba5c069d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0aa000 tx=0x7fba6c0b0040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:26.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:26.893+0000 7fba78ff9640 1 -- 172.21.15.52:0/2182363650 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fba7c161630 con 0x7fba7c1611b0 2024-05-04T12:19:27.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.053+0000 7fba81b58640 1 -- 172.21.15.52:0/2182363650 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba7c075320 con 0x7fba5c0678c0 2024-05-04T12:19:27.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.061+0000 7fba78ff9640 1 -- 172.21.15.52:0/2182363650 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251952 (secure 0 0 0) 0x7fba6c0f9840 con 0x7fba5c0678c0 2024-05-04T12:19:27.067 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:27.069 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-04T12:19:27.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.065+0000 7fba81b58640 1 -- 172.21.15.52:0/2182363650 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fba5c0678c0 msgr2=0x7fba5c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.065+0000 7fba81b58640 1 --2- 172.21.15.52:0/2182363650 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fba5c0678c0 0x7fba5c069d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0aa000 tx=0x7fba6c0b0040 comp rx=0 tx=0).stop 2024-05-04T12:19:27.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.065+0000 7fba81b58640 1 -- 172.21.15.52:0/2182363650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba7c1611b0 msgr2=0x7fba7c153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.065+0000 7fba81b58640 1 --2- 172.21.15.52:0/2182363650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba7c1611b0 0x7fba7c153550 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fba640108f0 tx=0x7fba64010db0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.065+0000 7fba81b58640 1 -- 172.21.15.52:0/2182363650 shutdown_connections 2024-05-04T12:19:27.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.065+0000 7fba81b58640 1 --2- 172.21.15.52:0/2182363650 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fba5c0678c0 0x7fba5c069d80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.065+0000 7fba81b58640 1 --2- 172.21.15.52:0/2182363650 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba7c14b200 0x7fba7c150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.065+0000 7fba81b58640 1 --2- 172.21.15.52:0/2182363650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba7c1611b0 0x7fba7c153550 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.065+0000 7fba81b58640 1 --2- 172.21.15.52:0/2182363650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba680039c0 0x7fba7c14fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.065+0000 7fba81b58640 1 -- 172.21.15.52:0/2182363650 >> 172.21.15.52:0/2182363650 conn(0x7fba7c0b14a0 msgr2=0x7fba7c15b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:27.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.065+0000 7fba81b58640 1 -- 172.21.15.52:0/2182363650 shutdown_connections 2024-05-04T12:19:27.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.065+0000 7fba81b58640 1 -- 172.21.15.52:0/2182363650 wait complete. 2024-05-04T12:19:27.086 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-05-04T12:19:26.758502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237048,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737672,"statfs":{"total":773094113280,"available":772851376128,"internally_reserved":0,"allocated":4333568,"data_stored":2397148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001691"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.648266+0000","last_change":"2024-05-04T12:19:22.684700+0000","last_active":"2024-05-04T12:19:23.648266+0000","last_peered":"2024-05-04T12:19:23.648266+0000","last_clean":"2024-05-04T12:19:23.648266+0000","last_became_active":"2024-05-04T12:19:22.683953+0000","last_became_peered":"2024-05-04T12:19:22.683953+0000","last_unstale":"2024-05-04T12:19:23.648266+0000","last_undegraded":"2024-05-04T12:19:23.648266+0000","last_fullsized":"2024-05-04T12:19:23.648266+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:53:38.859532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.646878+0000","last_change":"2024-05-04T12:19:22.683072+0000","last_active":"2024-05-04T12:19:23.646878+0000","last_peered":"2024-05-04T12:19:23.646878+0000","last_clean":"2024-05-04T12:19:23.646878+0000","last_became_active":"2024-05-04T12:19:22.682736+0000","last_became_peered":"2024-05-04T12:19:22.682736+0000","last_unstale":"2024-05-04T12:19:23.646878+0000","last_undegraded":"2024-05-04T12:19:23.646878+0000","last_fullsized":"2024-05-04T12:19:23.646878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:49:04.563709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.811317+0000","last_change":"2024-05-04T12:19:22.676558+0000","last_active":"2024-05-04T12:19:23.811317+0000","last_peered":"2024-05-04T12:19:23.811317+0000","last_clean":"2024-05-04T12:19:23.811317+0000","last_became_active":"2024-05-04T12:19:22.675801+0000","last_became_peered":"2024-05-04T12:19:22.675801+0000","last_unstale":"2024-05-04T12:19:23.811317+0000","last_undegraded":"2024-05-04T12:19:23.811317+0000","last_fullsized":"2024-05-04T12:19:23.811317+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T15:49:04.563709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.687374+0000","last_change":"2024-05-04T12:19:22.686313+0000","last_active":"2024-05-04T12:19:23.687374+0000","last_peered":"2024-05-04T12:19:23.687374+0000","last_clean":"2024-05-04T12:19:23.687374+0000","last_became_active":"2024-05-04T12:19:22.685956+0000","last_became_peered":"2024-05-04T12:19:22.685956+0000","last_unstale":"2024-05-04T12:19:23.687374+0000","last_undegraded":"2024-05-04T12:19:23.687374+0000","last_fullsized":"2024-05-04T12:19:23.687374+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:51:14.638471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.808537+0000","last_change":"2024-05-04T12:19:22.671890+0000","last_active":"2024-05-04T12:19:23.808537+0000","last_peered":"2024-05-04T12:19:23.808537+0000","last_clean":"2024-05-04T12:19:23.808537+0000","last_became_active":"2024-05-04T12:19:22.671020+0000","last_became_peered":"2024-05-04T12:19:22.671020+0000","last_unstale":"2024-05-04T12:19:23.808537+0000","last_undegraded":"2024-05-04T12:19:23.808537+0000","last_fullsized":"2024-05-04T12:19:23.808537+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:03:21.271608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.645487+0000","last_change":"2024-05-04T12:19:22.683863+0000","last_active":"2024-05-04T12:19:23.645487+0000","last_peered":"2024-05-04T12:19:23.645487+0000","last_clean":"2024-05-04T12:19:23.645487+0000","last_became_active":"2024-05-04T12:19:22.683505+0000","last_became_peered":"2024-05-04T12:19:22.683505+0000","last_unstale":"2024-05-04T12:19:23.645487+0000","last_undegraded":"2024-05-04T12:19:23.645487+0000","last_fullsized":"2024-05-04T12:19:23.645487+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:32:44.970867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.649255+0000","last_change":"2024-05-04T12:19:22.709087+0000","last_active":"2024-05-04T12:19:23.649255+0000","last_peered":"2024-05-04T12:19:23.649255+0000","last_clean":"2024-05-04T12:19:23.649255+0000","last_became_active":"2024-05-04T12:19:22.708484+0000","last_became_peered":"2024-05-04T12:19:22.708484+0000","last_unstale":"2024-05-04T12:19:23.649255+0000","last_undegraded":"2024-05-04T12:19:23.649255+0000","last_fullsized":"2024-05-04T12:19:23.649255+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:27:37.842512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.647537+0000","last_change":"2024-05-04T12:19:22.708446+0000","last_active":"2024-05-04T12:19:23.647537+0000","last_peered":"2024-05-04T12:19:23.647537+0000","last_clean":"2024-05-04T12:19:23.647537+0000","last_became_active":"2024-05-04T12:19:22.707867+0000","last_became_peered":"2024-05-04T12:19:22.707867+0000","last_unstale":"2024-05-04T12:19:23.647537+0000","last_undegraded":"2024-05-04T12:19:23.647537+0000","last_fullsized":"2024-05-04T12:19:23.647537+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:45:24.577823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.809699+0000","last_change":"2024-05-04T12:19:22.678238+0000","last_active":"2024-05-04T12:19:23.809699+0000","last_peered":"2024-05-04T12:19:23.809699+0000","last_clean":"2024-05-04T12:19:23.809699+0000","last_became_active":"2024-05-04T12:19:22.677491+0000","last_became_peered":"2024-05-04T12:19:22.677491+0000","last_unstale":"2024-05-04T12:19:23.809699+0000","last_undegraded":"2024-05-04T12:19:23.809699+0000","last_fullsized":"2024-05-04T12:19:23.809699+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T17:58:09.615286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.687570+0000","last_change":"2024-05-04T12:19:22.671723+0000","last_active":"2024-05-04T12:19:23.687570+0000","last_peered":"2024-05-04T12:19:23.687570+0000","last_clean":"2024-05-04T12:19:23.687570+0000","last_became_active":"2024-05-04T12:19:22.671004+0000","last_became_peered":"2024-05-04T12:19:22.671004+0000","last_unstale":"2024-05-04T12:19:23.687570+0000","last_undegraded":"2024-05-04T12:19:23.687570+0000","last_fullsized":"2024-05-04T12:19:23.687570+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:45:24.577823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.649516+0000","last_change":"2024-05-04T12:19:22.696871+0000","last_active":"2024-05-04T12:19:23.649516+0000","last_peered":"2024-05-04T12:19:23.649516+0000","last_clean":"2024-05-04T12:19:23.649516+0000","last_became_active":"2024-05-04T12:19:22.696153+0000","last_became_peered":"2024-05-04T12:19:22.696153+0000","last_unstale":"2024-05-04T12:19:23.649516+0000","last_undegraded":"2024-05-04T12:19:23.649516+0000","last_fullsized":"2024-05-04T12:19:23.649516+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:01:32.114606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.810981+0000","last_change":"2024-05-04T12:19:22.667561+0000","last_active":"2024-05-04T12:19:23.810981+0000","last_peered":"2024-05-04T12:19:23.810981+0000","last_clean":"2024-05-04T12:19:23.810981+0000","last_became_active":"2024-05-04T12:19:22.666719+0000","last_became_peered":"2024-05-04T12:19:22.666719+0000","last_unstale":"2024-05-04T12:19:23.810981+0000","last_undegraded":"2024-05-04T12:19:23.810981+0000","last_fullsized":"2024-05-04T12:19:23.810981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:48:17.598940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.808624+0000","last_change":"2024-05-04T12:19:22.684648+0000","last_active":"2024-05-04T12:19:23.808624+0000","last_peered":"2024-05-04T12:19:23.808624+0000","last_clean":"2024-05-04T12:19:23.808624+0000","last_became_active":"2024-05-04T12:19:22.682036+0000","last_became_peered":"2024-05-04T12:19:22.682036+0000","last_unstale":"2024-05-04T12:19:23.808624+0000","last_undegraded":"2024-05-04T12:19:23.808624+0000","last_fullsized":"2024-05-04T12:19:23.808624+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:36:51.620641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.646476+0000","last_change":"2024-05-04T12:19:22.683187+0000","last_active":"2024-05-04T12:19:23.646476+0000","last_peered":"2024-05-04T12:19:23.646476+0000","last_clean":"2024-05-04T12:19:23.646476+0000","last_became_active":"2024-05-04T12:19:22.682604+0000","last_became_peered":"2024-05-04T12:19:22.682604+0000","last_unstale":"2024-05-04T12:19:23.646476+0000","last_undegraded":"2024-05-04T12:19:23.646476+0000","last_fullsized":"2024-05-04T12:19:23.646476+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:13:51.860759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.810656+0000","last_change":"2024-05-04T12:19:22.674177+0000","last_active":"2024-05-04T12:19:23.810656+0000","last_peered":"2024-05-04T12:19:23.810656+0000","last_clean":"2024-05-04T12:19:23.810656+0000","last_became_active":"2024-05-04T12:19:22.673479+0000","last_became_peered":"2024-05-04T12:19:22.673479+0000","last_unstale":"2024-05-04T12:19:23.810656+0000","last_undegraded":"2024-05-04T12:19:23.810656+0000","last_fullsized":"2024-05-04T12:19:23.810656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:32:44.970867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.649429+0000","last_change":"2024-05-04T12:19:22.701094+0000","last_active":"2024-05-04T12:19:23.649429+0000","last_peered":"2024-05-04T12:19:23.649429+0000","last_clean":"2024-05-04T12:19:23.649429+0000","last_became_active":"2024-05-04T12:19:22.699933+0000","last_became_peered":"2024-05-04T12:19:22.699933+0000","last_unstale":"2024-05-04T12:19:23.649429+0000","last_undegraded":"2024-05-04T12:19:23.649429+0000","last_fullsized":"2024-05-04T12:19:23.649429+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:48:17.598940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.649254+0000","last_change":"2024-05-04T12:19:22.709641+0000","last_active":"2024-05-04T12:19:23.649254+0000","last_peered":"2024-05-04T12:19:23.649254+0000","last_clean":"2024-05-04T12:19:23.649254+0000","last_became_active":"2024-05-04T12:19:22.709082+0000","last_became_peered":"2024-05-04T12:19:22.709082+0000","last_unstale":"2024-05-04T12:19:23.649254+0000","last_undegraded":"2024-05-04T12:19:23.649254+0000","last_fullsized":"2024-05-04T12:19:23.649254+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:48:36.580306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.647018+0000","last_change":"2024-05-04T12:19:22.682524+0000","last_active":"2024-05-04T12:19:23.647018+0000","last_peered":"2024-05-04T12:19:23.647018+0000","last_clean":"2024-05-04T12:19:23.647018+0000","last_became_active":"2024-05-04T12:19:22.681382+0000","last_became_peered":"2024-05-04T12:19:22.681382+0000","last_unstale":"2024-05-04T12:19:23.647018+0000","last_undegraded":"2024-05-04T12:19:23.647018+0000","last_fullsized":"2024-05-04T12:19:23.647018+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:16:00.206881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.645838+0000","last_change":"2024-05-04T12:19:22.683062+0000","last_active":"2024-05-04T12:19:23.645838+0000","last_peered":"2024-05-04T12:19:23.645838+0000","last_clean":"2024-05-04T12:19:23.645838+0000","last_became_active":"2024-05-04T12:19:22.682636+0000","last_became_peered":"2024-05-04T12:19:22.682636+0000","last_unstale":"2024-05-04T12:19:23.645838+0000","last_undegraded":"2024-05-04T12:19:23.645838+0000","last_fullsized":"2024-05-04T12:19:23.645838+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:48:36.580306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.808954+0000","last_change":"2024-05-04T12:19:22.677444+0000","last_active":"2024-05-04T12:19:23.808954+0000","last_peered":"2024-05-04T12:19:23.808954+0000","last_clean":"2024-05-04T12:19:23.808954+0000","last_became_active":"2024-05-04T12:19:22.677105+0000","last_became_peered":"2024-05-04T12:19:22.677105+0000","last_unstale":"2024-05-04T12:19:23.808954+0000","last_undegraded":"2024-05-04T12:19:23.808954+0000","last_fullsized":"2024-05-04T12:19:23.808954+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T17:36:38.387547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.809228+0000","last_change":"2024-05-04T12:19:22.676351+0000","last_active":"2024-05-04T12:19:23.809228+0000","last_peered":"2024-05-04T12:19:23.809228+0000","last_clean":"2024-05-04T12:19:23.809228+0000","last_became_active":"2024-05-04T12:19:22.675384+0000","last_became_peered":"2024-05-04T12:19:22.675384+0000","last_unstale":"2024-05-04T12:19:23.809228+0000","last_undegraded":"2024-05-04T12:19:23.809228+0000","last_fullsized":"2024-05-04T12:19:23.809228+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:11:40.771964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.809208+0000","last_change":"2024-05-04T12:19:22.680150+0000","last_active":"2024-05-04T12:19:23.809208+0000","last_peered":"2024-05-04T12:19:23.809208+0000","last_clean":"2024-05-04T12:19:23.809208+0000","last_became_active":"2024-05-04T12:19:22.676709+0000","last_became_peered":"2024-05-04T12:19:22.676709+0000","last_unstale":"2024-05-04T12:19:23.809208+0000","last_undegraded":"2024-05-04T12:19:23.809208+0000","last_fullsized":"2024-05-04T12:19:23.809208+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:16:06.107860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.683833+0000","last_change":"2024-05-04T12:19:22.695584+0000","last_active":"2024-05-04T12:19:23.683833+0000","last_peered":"2024-05-04T12:19:23.683833+0000","last_clean":"2024-05-04T12:19:23.683833+0000","last_became_active":"2024-05-04T12:19:22.695259+0000","last_became_peered":"2024-05-04T12:19:22.695259+0000","last_unstale":"2024-05-04T12:19:23.683833+0000","last_undegraded":"2024-05-04T12:19:23.683833+0000","last_fullsized":"2024-05-04T12:19:23.683833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:42:39.061315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.809069+0000","last_change":"2024-05-04T12:19:22.662107+0000","last_active":"2024-05-04T12:19:23.809069+0000","last_peered":"2024-05-04T12:19:23.809069+0000","last_clean":"2024-05-04T12:19:23.809069+0000","last_became_active":"2024-05-04T12:19:22.661315+0000","last_became_peered":"2024-05-04T12:19:22.661315+0000","last_unstale":"2024-05-04T12:19:23.809069+0000","last_undegraded":"2024-05-04T12:19:23.809069+0000","last_fullsized":"2024-05-04T12:19:23.809069+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T17:32:05.053202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.687850+0000","last_change":"2024-05-04T12:19:22.703116+0000","last_active":"2024-05-04T12:19:23.687850+0000","last_peered":"2024-05-04T12:19:23.687850+0000","last_clean":"2024-05-04T12:19:23.687850+0000","last_became_active":"2024-05-04T12:19:22.702832+0000","last_became_peered":"2024-05-04T12:19:22.702832+0000","last_unstale":"2024-05-04T12:19:23.687850+0000","last_undegraded":"2024-05-04T12:19:23.687850+0000","last_fullsized":"2024-05-04T12:19:23.687850+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:49:28.269102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.647238+0000","last_change":"2024-05-04T12:19:22.697816+0000","last_active":"2024-05-04T12:19:23.647238+0000","last_peered":"2024-05-04T12:19:23.647238+0000","last_clean":"2024-05-04T12:19:23.647238+0000","last_became_active":"2024-05-04T12:19:22.697297+0000","last_became_peered":"2024-05-04T12:19:22.697297+0000","last_unstale":"2024-05-04T12:19:23.647238+0000","last_undegraded":"2024-05-04T12:19:23.647238+0000","last_fullsized":"2024-05-04T12:19:23.647238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:57:51.506509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.687931+0000","last_change":"2024-05-04T12:19:22.682229+0000","last_active":"2024-05-04T12:19:23.687931+0000","last_peered":"2024-05-04T12:19:23.687931+0000","last_clean":"2024-05-04T12:19:23.687931+0000","last_became_active":"2024-05-04T12:19:22.681858+0000","last_became_peered":"2024-05-04T12:19:22.681858+0000","last_unstale":"2024-05-04T12:19:23.687931+0000","last_undegraded":"2024-05-04T12:19:23.687931+0000","last_fullsized":"2024-05-04T12:19:23.687931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:48:59.736951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.811004+0000","last_change":"2024-05-04T12:19:22.668934+0000","last_active":"2024-05-04T12:19:23.811004+0000","last_peered":"2024-05-04T12:19:23.811004+0000","last_clean":"2024-05-04T12:19:23.811004+0000","last_became_active":"2024-05-04T12:19:22.667461+0000","last_became_peered":"2024-05-04T12:19:22.667461+0000","last_unstale":"2024-05-04T12:19:23.811004+0000","last_undegraded":"2024-05-04T12:19:23.811004+0000","last_fullsized":"2024-05-04T12:19:23.811004+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:01:32.114606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.809565+0000","last_change":"2024-05-04T12:19:22.671958+0000","last_active":"2024-05-04T12:19:23.809565+0000","last_peered":"2024-05-04T12:19:23.809565+0000","last_clean":"2024-05-04T12:19:23.809565+0000","last_became_active":"2024-05-04T12:19:22.671172+0000","last_became_peered":"2024-05-04T12:19:22.671172+0000","last_unstale":"2024-05-04T12:19:23.809565+0000","last_undegraded":"2024-05-04T12:19:23.809565+0000","last_fullsized":"2024-05-04T12:19:23.809565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:51:14.638471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.648723+0000","last_change":"2024-05-04T12:19:22.696605+0000","last_active":"2024-05-04T12:19:23.648723+0000","last_peered":"2024-05-04T12:19:23.648723+0000","last_clean":"2024-05-04T12:19:23.648723+0000","last_became_active":"2024-05-04T12:19:22.695838+0000","last_became_peered":"2024-05-04T12:19:22.695838+0000","last_unstale":"2024-05-04T12:19:23.648723+0000","last_undegraded":"2024-05-04T12:19:23.648723+0000","last_fullsized":"2024-05-04T12:19:23.648723+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:48:59.736951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.647055+0000","last_change":"2024-05-04T12:19:22.679303+0000","last_active":"2024-05-04T12:19:23.647055+0000","last_peered":"2024-05-04T12:19:23.647055+0000","last_clean":"2024-05-04T12:19:23.647055+0000","last_became_active":"2024-05-04T12:19:22.678529+0000","last_became_peered":"2024-05-04T12:19:22.678529+0000","last_unstale":"2024-05-04T12:19:23.647055+0000","last_undegraded":"2024-05-04T12:19:23.647055+0000","last_fullsized":"2024-05-04T12:19:23.647055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T17:36:38.387547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.647931+0000","last_change":"2024-05-04T12:19:22.700491+0000","last_active":"2024-05-04T12:19:23.647931+0000","last_peered":"2024-05-04T12:19:23.647931+0000","last_clean":"2024-05-04T12:19:23.647931+0000","last_became_active":"2024-05-04T12:19:22.697347+0000","last_became_peered":"2024-05-04T12:19:22.697347+0000","last_unstale":"2024-05-04T12:19:23.647931+0000","last_undegraded":"2024-05-04T12:19:23.647931+0000","last_fullsized":"2024-05-04T12:19:23.647931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:18:10.437588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.684050+0000","last_change":"2024-05-04T12:19:22.695588+0000","last_active":"2024-05-04T12:19:23.684050+0000","last_peered":"2024-05-04T12:19:23.684050+0000","last_clean":"2024-05-04T12:19:23.684050+0000","last_became_active":"2024-05-04T12:19:22.695189+0000","last_became_peered":"2024-05-04T12:19:22.695189+0000","last_unstale":"2024-05-04T12:19:23.684050+0000","last_undegraded":"2024-05-04T12:19:23.684050+0000","last_fullsized":"2024-05-04T12:19:23.684050+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T22:27:24.827385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.648962+0000","last_change":"2024-05-04T12:19:21.678455+0000","last_active":"2024-05-04T12:19:23.648962+0000","last_peered":"2024-05-04T12:19:23.648962+0000","last_clean":"2024-05-04T12:19:23.648962+0000","last_became_active":"2024-05-04T12:19:21.677670+0000","last_became_peered":"2024-05-04T12:19:21.677670+0000","last_unstale":"2024-05-04T12:19:23.648962+0000","last_undegraded":"2024-05-04T12:19:23.648962+0000","last_fullsized":"2024-05-04T12:19:23.648962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:20:07.582246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.648651+0000","last_change":"2024-05-04T12:19:22.671115+0000","last_active":"2024-05-04T12:19:23.648651+0000","last_peered":"2024-05-04T12:19:23.648651+0000","last_clean":"2024-05-04T12:19:23.648651+0000","last_became_active":"2024-05-04T12:19:22.670527+0000","last_became_peered":"2024-05-04T12:19:22.670527+0000","last_unstale":"2024-05-04T12:19:23.648651+0000","last_undegraded":"2024-05-04T12:19:23.648651+0000","last_fullsized":"2024-05-04T12:19:23.648651+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:58:14.561076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.684278+0000","last_change":"2024-05-04T12:19:21.666457+0000","last_active":"2024-05-04T12:19:23.684278+0000","last_peered":"2024-05-04T12:19:23.684278+0000","last_clean":"2024-05-04T12:19:23.684278+0000","last_became_active":"2024-05-04T12:19:21.666088+0000","last_became_peered":"2024-05-04T12:19:21.666088+0000","last_unstale":"2024-05-04T12:19:23.684278+0000","last_undegraded":"2024-05-04T12:19:23.684278+0000","last_fullsized":"2024-05-04T12:19:23.684278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:36:50.612240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.810265+0000","last_change":"2024-05-04T12:19:22.678692+0000","last_active":"2024-05-04T12:19:23.810265+0000","last_peered":"2024-05-04T12:19:23.810265+0000","last_clean":"2024-05-04T12:19:23.810265+0000","last_became_active":"2024-05-04T12:19:22.676944+0000","last_became_peered":"2024-05-04T12:19:22.676944+0000","last_unstale":"2024-05-04T12:19:23.810265+0000","last_undegraded":"2024-05-04T12:19:23.810265+0000","last_fullsized":"2024-05-04T12:19:23.810265+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:18:10.437588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.646525+0000","last_change":"2024-05-04T12:19:21.657118+0000","last_active":"2024-05-04T12:19:23.646525+0000","last_peered":"2024-05-04T12:19:23.646525+0000","last_clean":"2024-05-04T12:19:23.646525+0000","last_became_active":"2024-05-04T12:19:21.656703+0000","last_became_peered":"2024-05-04T12:19:21.656703+0000","last_unstale":"2024-05-04T12:19:23.646525+0000","last_undegraded":"2024-05-04T12:19:23.646525+0000","last_fullsized":"2024-05-04T12:19:23.646525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:53:37.851131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.687795+0000","last_change":"2024-05-04T12:19:22.659989+0000","last_active":"2024-05-04T12:19:23.687795+0000","last_peered":"2024-05-04T12:19:23.687795+0000","last_clean":"2024-05-04T12:19:23.687795+0000","last_became_active":"2024-05-04T12:19:22.659223+0000","last_became_peered":"2024-05-04T12:19:22.659223+0000","last_unstale":"2024-05-04T12:19:23.687795+0000","last_undegraded":"2024-05-04T12:19:23.687795+0000","last_fullsized":"2024-05-04T12:19:23.687795+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:18:10.437588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.808791+0000","last_change":"2024-05-04T12:19:21.659138+0000","last_active":"2024-05-04T12:19:23.808791+0000","last_peered":"2024-05-04T12:19:23.808791+0000","last_clean":"2024-05-04T12:19:23.808791+0000","last_became_active":"2024-05-04T12:19:21.658805+0000","last_became_peered":"2024-05-04T12:19:21.658805+0000","last_unstale":"2024-05-04T12:19:23.808791+0000","last_undegraded":"2024-05-04T12:19:23.808791+0000","last_fullsized":"2024-05-04T12:19:23.808791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:10:39.271254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.649238+0000","last_change":"2024-05-04T12:19:22.702536+0000","last_active":"2024-05-04T12:19:23.649238+0000","last_peered":"2024-05-04T12:19:23.649238+0000","last_clean":"2024-05-04T12:19:23.649238+0000","last_became_active":"2024-05-04T12:19:22.701811+0000","last_became_peered":"2024-05-04T12:19:22.701811+0000","last_unstale":"2024-05-04T12:19:23.649238+0000","last_undegraded":"2024-05-04T12:19:23.649238+0000","last_fullsized":"2024-05-04T12:19:23.649238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:32:44.970867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.809519+0000","last_change":"2024-05-04T12:19:21.678189+0000","last_active":"2024-05-04T12:19:23.809519+0000","last_peered":"2024-05-04T12:19:23.809519+0000","last_clean":"2024-05-04T12:19:23.809519+0000","last_became_active":"2024-05-04T12:19:21.677433+0000","last_became_peered":"2024-05-04T12:19:21.677433+0000","last_unstale":"2024-05-04T12:19:23.809519+0000","last_undegraded":"2024-05-04T12:19:23.809519+0000","last_fullsized":"2024-05-04T12:19:23.809519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:36:50.612240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.810863+0000","last_change":"2024-05-04T12:19:22.682397+0000","last_active":"2024-05-04T12:19:23.810863+0000","last_peered":"2024-05-04T12:19:23.810863+0000","last_clean":"2024-05-04T12:19:23.810863+0000","last_became_active":"2024-05-04T12:19:22.681641+0000","last_became_peered":"2024-05-04T12:19:22.681641+0000","last_unstale":"2024-05-04T12:19:23.810863+0000","last_undegraded":"2024-05-04T12:19:23.810863+0000","last_fullsized":"2024-05-04T12:19:23.810863+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:48:36.580306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.809490+0000","last_change":"2024-05-04T12:19:21.662829+0000","last_active":"2024-05-04T12:19:23.809490+0000","last_peered":"2024-05-04T12:19:23.809490+0000","last_clean":"2024-05-04T12:19:23.809490+0000","last_became_active":"2024-05-04T12:19:21.661888+0000","last_became_peered":"2024-05-04T12:19:21.661888+0000","last_unstale":"2024-05-04T12:19:23.809490+0000","last_undegraded":"2024-05-04T12:19:23.809490+0000","last_fullsized":"2024-05-04T12:19:23.809490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:03:20.263207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.809121+0000","last_change":"2024-05-04T12:19:22.679531+0000","last_active":"2024-05-04T12:19:23.809121+0000","last_peered":"2024-05-04T12:19:23.809121+0000","last_clean":"2024-05-04T12:19:23.809121+0000","last_became_active":"2024-05-04T12:19:22.678729+0000","last_became_peered":"2024-05-04T12:19:22.678729+0000","last_unstale":"2024-05-04T12:19:23.809121+0000","last_undegraded":"2024-05-04T12:19:23.809121+0000","last_fullsized":"2024-05-04T12:19:23.809121+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T17:52:39.897158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.645472+0000","last_change":"2024-05-04T12:19:21.653368+0000","last_active":"2024-05-04T12:19:23.645472+0000","last_peered":"2024-05-04T12:19:23.645472+0000","last_clean":"2024-05-04T12:19:23.645472+0000","last_became_active":"2024-05-04T12:19:21.653034+0000","last_became_peered":"2024-05-04T12:19:21.653034+0000","last_unstale":"2024-05-04T12:19:23.645472+0000","last_undegraded":"2024-05-04T12:19:23.645472+0000","last_fullsized":"2024-05-04T12:19:23.645472+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T22:47:32.709430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.687023+0000","last_change":"2024-05-04T12:19:22.678894+0000","last_active":"2024-05-04T12:19:23.687023+0000","last_peered":"2024-05-04T12:19:23.687023+0000","last_clean":"2024-05-04T12:19:23.687023+0000","last_became_active":"2024-05-04T12:19:22.675789+0000","last_became_peered":"2024-05-04T12:19:22.675789+0000","last_unstale":"2024-05-04T12:19:23.687023+0000","last_undegraded":"2024-05-04T12:19:23.687023+0000","last_fullsized":"2024-05-04T12:19:23.687023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:16:06.107860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.809393+0000","last_change":"2024-05-04T12:19:21.670006+0000","last_active":"2024-05-04T12:19:23.809393+0000","last_peered":"2024-05-04T12:19:23.809393+0000","last_clean":"2024-05-04T12:19:23.809393+0000","last_became_active":"2024-05-04T12:19:21.669224+0000","last_became_peered":"2024-05-04T12:19:21.669224+0000","last_unstale":"2024-05-04T12:19:23.809393+0000","last_undegraded":"2024-05-04T12:19:23.809393+0000","last_fullsized":"2024-05-04T12:19:23.809393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:34:53.245509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.811260+0000","last_change":"2024-05-04T12:19:22.677231+0000","last_active":"2024-05-04T12:19:23.811260+0000","last_peered":"2024-05-04T12:19:23.811260+0000","last_clean":"2024-05-04T12:19:23.811260+0000","last_became_active":"2024-05-04T12:19:22.676528+0000","last_became_peered":"2024-05-04T12:19:22.676528+0000","last_unstale":"2024-05-04T12:19:23.811260+0000","last_undegraded":"2024-05-04T12:19:23.811260+0000","last_fullsized":"2024-05-04T12:19:23.811260+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:13:51.860759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.645955+0000","last_change":"2024-05-04T12:19:21.667225+0000","last_active":"2024-05-04T12:19:23.645955+0000","last_peered":"2024-05-04T12:19:23.645955+0000","last_clean":"2024-05-04T12:19:23.645955+0000","last_became_active":"2024-05-04T12:19:21.666940+0000","last_became_peered":"2024-05-04T12:19:21.666940+0000","last_unstale":"2024-05-04T12:19:23.645955+0000","last_undegraded":"2024-05-04T12:19:23.645955+0000","last_fullsized":"2024-05-04T12:19:23.645955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:58:13.552675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.645855+0000","last_change":"2024-05-04T12:19:22.682798+0000","last_active":"2024-05-04T12:19:23.645855+0000","last_peered":"2024-05-04T12:19:23.645855+0000","last_clean":"2024-05-04T12:19:23.645855+0000","last_became_active":"2024-05-04T12:19:22.681454+0000","last_became_peered":"2024-05-04T12:19:22.681454+0000","last_unstale":"2024-05-04T12:19:23.645855+0000","last_undegraded":"2024-05-04T12:19:23.645855+0000","last_fullsized":"2024-05-04T12:19:23.645855+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:27:37.842512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.684301+0000","last_change":"2024-05-04T12:19:21.659765+0000","last_active":"2024-05-04T12:19:23.684301+0000","last_peered":"2024-05-04T12:19:23.684301+0000","last_clean":"2024-05-04T12:19:23.684301+0000","last_became_active":"2024-05-04T12:19:21.659504+0000","last_became_peered":"2024-05-04T12:19:21.659504+0000","last_unstale":"2024-05-04T12:19:23.684301+0000","last_undegraded":"2024-05-04T12:19:23.684301+0000","last_fullsized":"2024-05-04T12:19:23.684301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:30:49.875995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.646301+0000","last_change":"2024-05-04T12:19:22.699423+0000","last_active":"2024-05-04T12:19:23.646301+0000","last_peered":"2024-05-04T12:19:23.646301+0000","last_clean":"2024-05-04T12:19:23.646301+0000","last_became_active":"2024-05-04T12:19:22.698196+0000","last_became_peered":"2024-05-04T12:19:22.698196+0000","last_unstale":"2024-05-04T12:19:23.646301+0000","last_undegraded":"2024-05-04T12:19:23.646301+0000","last_fullsized":"2024-05-04T12:19:23.646301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:01:32.114606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.811295+0000","last_change":"2024-05-04T12:19:21.680749+0000","last_active":"2024-05-04T12:19:23.811295+0000","last_peered":"2024-05-04T12:19:23.811295+0000","last_clean":"2024-05-04T12:19:23.811295+0000","last_became_active":"2024-05-04T12:19:21.678635+0000","last_became_peered":"2024-05-04T12:19:21.678635+0000","last_unstale":"2024-05-04T12:19:23.811295+0000","last_undegraded":"2024-05-04T12:19:23.811295+0000","last_fullsized":"2024-05-04T12:19:23.811295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:39:58.703614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.810947+0000","last_change":"2024-05-04T12:19:22.684849+0000","last_active":"2024-05-04T12:19:23.810947+0000","last_peered":"2024-05-04T12:19:23.810947+0000","last_clean":"2024-05-04T12:19:23.810947+0000","last_became_active":"2024-05-04T12:19:22.684142+0000","last_became_peered":"2024-05-04T12:19:22.684142+0000","last_unstale":"2024-05-04T12:19:23.810947+0000","last_undegraded":"2024-05-04T12:19:23.810947+0000","last_fullsized":"2024-05-04T12:19:23.810947+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:27:37.842512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.646346+0000","last_change":"2024-05-04T12:19:21.660271+0000","last_active":"2024-05-04T12:19:23.646346+0000","last_peered":"2024-05-04T12:19:23.646346+0000","last_clean":"2024-05-04T12:19:23.646346+0000","last_became_active":"2024-05-04T12:19:21.659682+0000","last_became_peered":"2024-05-04T12:19:21.659682+0000","last_unstale":"2024-05-04T12:19:23.646346+0000","last_undegraded":"2024-05-04T12:19:23.646346+0000","last_fullsized":"2024-05-04T12:19:23.646346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:20:04.956887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.687136+0000","last_change":"2024-05-04T12:19:22.675737+0000","last_active":"2024-05-04T12:19:23.687136+0000","last_peered":"2024-05-04T12:19:23.687136+0000","last_clean":"2024-05-04T12:19:23.687136+0000","last_became_active":"2024-05-04T12:19:22.675086+0000","last_became_peered":"2024-05-04T12:19:22.675086+0000","last_unstale":"2024-05-04T12:19:23.687136+0000","last_undegraded":"2024-05-04T12:19:23.687136+0000","last_fullsized":"2024-05-04T12:19:23.687136+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:11:40.771964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.809808+0000","last_change":"2024-05-04T12:19:21.681112+0000","last_active":"2024-05-04T12:19:23.809808+0000","last_peered":"2024-05-04T12:19:23.809808+0000","last_clean":"2024-05-04T12:19:23.809808+0000","last_became_active":"2024-05-04T12:19:21.679031+0000","last_became_peered":"2024-05-04T12:19:21.679031+0000","last_unstale":"2024-05-04T12:19:23.809808+0000","last_undegraded":"2024-05-04T12:19:23.809808+0000","last_fullsized":"2024-05-04T12:19:23.809808+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:43:24.099486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.646184+0000","last_change":"2024-05-04T12:19:22.683261+0000","last_active":"2024-05-04T12:19:23.646184+0000","last_peered":"2024-05-04T12:19:23.646184+0000","last_clean":"2024-05-04T12:19:23.646184+0000","last_became_active":"2024-05-04T12:19:22.682989+0000","last_became_peered":"2024-05-04T12:19:22.682989+0000","last_unstale":"2024-05-04T12:19:23.646184+0000","last_undegraded":"2024-05-04T12:19:23.646184+0000","last_fullsized":"2024-05-04T12:19:23.646184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:48:17.598940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.687303+0000","last_change":"2024-05-04T12:19:21.665602+0000","last_active":"2024-05-04T12:19:23.687303+0000","last_peered":"2024-05-04T12:19:23.687303+0000","last_clean":"2024-05-04T12:19:23.687303+0000","last_became_active":"2024-05-04T12:19:21.664708+0000","last_became_peered":"2024-05-04T12:19:21.664708+0000","last_unstale":"2024-05-04T12:19:23.687303+0000","last_undegraded":"2024-05-04T12:19:23.687303+0000","last_fullsized":"2024-05-04T12:19:23.687303+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:43:24.099486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.648827+0000","last_change":"2024-05-04T12:19:22.700612+0000","last_active":"2024-05-04T12:19:23.648827+0000","last_peered":"2024-05-04T12:19:23.648827+0000","last_clean":"2024-05-04T12:19:23.648827+0000","last_became_active":"2024-05-04T12:19:22.699663+0000","last_became_peered":"2024-05-04T12:19:22.699663+0000","last_unstale":"2024-05-04T12:19:23.648827+0000","last_undegraded":"2024-05-04T12:19:23.648827+0000","last_fullsized":"2024-05-04T12:19:23.648827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T17:32:05.053202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.645260+0000","last_change":"2024-05-04T12:19:21.667099+0000","last_active":"2024-05-04T12:19:23.645260+0000","last_peered":"2024-05-04T12:19:23.645260+0000","last_clean":"2024-05-04T12:19:23.645260+0000","last_became_active":"2024-05-04T12:19:21.666768+0000","last_became_peered":"2024-05-04T12:19:21.666768+0000","last_unstale":"2024-05-04T12:19:23.645260+0000","last_undegraded":"2024-05-04T12:19:23.645260+0000","last_fullsized":"2024-05-04T12:19:23.645260+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:12:22.175099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.647152+0000","last_change":"2024-05-04T12:19:22.669000+0000","last_active":"2024-05-04T12:19:23.647152+0000","last_peered":"2024-05-04T12:19:23.647152+0000","last_clean":"2024-05-04T12:19:23.647152+0000","last_became_active":"2024-05-04T12:19:22.668222+0000","last_became_peered":"2024-05-04T12:19:22.668222+0000","last_unstale":"2024-05-04T12:19:23.647152+0000","last_undegraded":"2024-05-04T12:19:23.647152+0000","last_fullsized":"2024-05-04T12:19:23.647152+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:12:23.183500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.645221+0000","last_change":"2024-05-04T12:19:21.657454+0000","last_active":"2024-05-04T12:19:23.645221+0000","last_peered":"2024-05-04T12:19:23.645221+0000","last_clean":"2024-05-04T12:19:23.645221+0000","last_became_active":"2024-05-04T12:19:21.656795+0000","last_became_peered":"2024-05-04T12:19:21.656795+0000","last_unstale":"2024-05-04T12:19:23.645221+0000","last_undegraded":"2024-05-04T12:19:23.645221+0000","last_fullsized":"2024-05-04T12:19:23.645221+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:42:38.052914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.809862+0000","last_change":"2024-05-04T12:19:22.658939+0000","last_active":"2024-05-04T12:19:23.809862+0000","last_peered":"2024-05-04T12:19:23.809862+0000","last_clean":"2024-05-04T12:19:23.809862+0000","last_became_active":"2024-05-04T12:19:22.658236+0000","last_became_peered":"2024-05-04T12:19:22.658236+0000","last_unstale":"2024-05-04T12:19:23.809862+0000","last_undegraded":"2024-05-04T12:19:23.809862+0000","last_fullsized":"2024-05-04T12:19:23.809862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:45:24.577823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.687964+0000","last_change":"2024-05-04T12:19:21.660618+0000","last_active":"2024-05-04T12:19:23.687964+0000","last_peered":"2024-05-04T12:19:23.687964+0000","last_clean":"2024-05-04T12:19:23.687964+0000","last_became_active":"2024-05-04T12:19:21.660223+0000","last_became_peered":"2024-05-04T12:19:21.660223+0000","last_unstale":"2024-05-04T12:19:23.687964+0000","last_undegraded":"2024-05-04T12:19:23.687964+0000","last_fullsized":"2024-05-04T12:19:23.687964+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T17:36:37.379146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.811812+0000","last_change":"2024-05-04T12:19:22.657935+0000","last_active":"2024-05-04T12:19:23.811812+0000","last_peered":"2024-05-04T12:19:23.811812+0000","last_clean":"2024-05-04T12:19:23.811812+0000","last_became_active":"2024-05-04T12:19:22.656923+0000","last_became_peered":"2024-05-04T12:19:22.656923+0000","last_unstale":"2024-05-04T12:19:23.811812+0000","last_undegraded":"2024-05-04T12:19:23.811812+0000","last_fullsized":"2024-05-04T12:19:23.811812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:16:00.206881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.809860+0000","last_change":"2024-05-04T12:19:21.677013+0000","last_active":"2024-05-04T12:19:23.809860+0000","last_peered":"2024-05-04T12:19:23.809860+0000","last_clean":"2024-05-04T12:19:23.809860+0000","last_became_active":"2024-05-04T12:19:21.669467+0000","last_became_peered":"2024-05-04T12:19:21.669467+0000","last_unstale":"2024-05-04T12:19:23.809860+0000","last_undegraded":"2024-05-04T12:19:23.809860+0000","last_fullsized":"2024-05-04T12:19:23.809860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:33:48.709731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.687551+0000","last_change":"2024-05-04T12:19:22.673329+0000","last_active":"2024-05-04T12:19:23.687551+0000","last_peered":"2024-05-04T12:19:23.687551+0000","last_clean":"2024-05-04T12:19:23.687551+0000","last_became_active":"2024-05-04T12:19:22.672594+0000","last_became_peered":"2024-05-04T12:19:22.672594+0000","last_unstale":"2024-05-04T12:19:23.687551+0000","last_undegraded":"2024-05-04T12:19:23.687551+0000","last_fullsized":"2024-05-04T12:19:23.687551+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T17:58:09.615286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.809840+0000","last_change":"2024-05-04T12:19:21.669229+0000","last_active":"2024-05-04T12:19:23.809840+0000","last_peered":"2024-05-04T12:19:23.809840+0000","last_clean":"2024-05-04T12:19:23.809840+0000","last_became_active":"2024-05-04T12:19:21.668411+0000","last_became_peered":"2024-05-04T12:19:21.668411+0000","last_unstale":"2024-05-04T12:19:23.809840+0000","last_undegraded":"2024-05-04T12:19:23.809840+0000","last_fullsized":"2024-05-04T12:19:23.809840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:20:07.582246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.808833+0000","last_change":"2024-05-04T12:19:22.697359+0000","last_active":"2024-05-04T12:19:23.808833+0000","last_peered":"2024-05-04T12:19:23.808833+0000","last_clean":"2024-05-04T12:19:23.808833+0000","last_became_active":"2024-05-04T12:19:22.697086+0000","last_became_peered":"2024-05-04T12:19:22.697086+0000","last_unstale":"2024-05-04T12:19:23.808833+0000","last_undegraded":"2024-05-04T12:19:23.808833+0000","last_fullsized":"2024-05-04T12:19:23.808833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:30:50.884396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.810501+0000","last_change":"2024-05-04T12:19:21.663997+0000","last_active":"2024-05-04T12:19:23.810501+0000","last_peered":"2024-05-04T12:19:23.810501+0000","last_clean":"2024-05-04T12:19:23.810501+0000","last_became_active":"2024-05-04T12:19:21.663246+0000","last_became_peered":"2024-05-04T12:19:21.663246+0000","last_unstale":"2024-05-04T12:19:23.810501+0000","last_undegraded":"2024-05-04T12:19:23.810501+0000","last_fullsized":"2024-05-04T12:19:23.810501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:30:49.875995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.808443+0000","last_change":"2024-05-04T12:19:22.668746+0000","last_active":"2024-05-04T12:19:23.808443+0000","last_peered":"2024-05-04T12:19:23.808443+0000","last_clean":"2024-05-04T12:19:23.808443+0000","last_became_active":"2024-05-04T12:19:22.665100+0000","last_became_peered":"2024-05-04T12:19:22.665100+0000","last_unstale":"2024-05-04T12:19:23.808443+0000","last_undegraded":"2024-05-04T12:19:23.808443+0000","last_fullsized":"2024-05-04T12:19:23.808443+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:33:49.718132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.684550+0000","last_change":"2024-05-04T12:19:21.659748+0000","last_active":"2024-05-04T12:19:23.684550+0000","last_peered":"2024-05-04T12:19:23.684550+0000","last_clean":"2024-05-04T12:19:23.684550+0000","last_became_active":"2024-05-04T12:19:21.659455+0000","last_became_peered":"2024-05-04T12:19:21.659455+0000","last_unstale":"2024-05-04T12:19:23.684550+0000","last_undegraded":"2024-05-04T12:19:23.684550+0000","last_fullsized":"2024-05-04T12:19:23.684550+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:12:22.175099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.808973+0000","last_change":"2024-05-04T12:19:22.680529+0000","last_active":"2024-05-04T12:19:23.808973+0000","last_peered":"2024-05-04T12:19:23.808973+0000","last_clean":"2024-05-04T12:19:23.808973+0000","last_became_active":"2024-05-04T12:19:22.680226+0000","last_became_peered":"2024-05-04T12:19:22.680226+0000","last_unstale":"2024-05-04T12:19:23.808973+0000","last_undegraded":"2024-05-04T12:19:23.808973+0000","last_fullsized":"2024-05-04T12:19:23.808973+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:12:23.183500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.650152+0000","last_change":"2024-05-04T12:19:21.676794+0000","last_active":"2024-05-04T12:19:23.650152+0000","last_peered":"2024-05-04T12:19:23.650152+0000","last_clean":"2024-05-04T12:19:23.650152+0000","last_became_active":"2024-05-04T12:19:21.675839+0000","last_became_peered":"2024-05-04T12:19:21.675839+0000","last_unstale":"2024-05-04T12:19:23.650152+0000","last_undegraded":"2024-05-04T12:19:23.650152+0000","last_fullsized":"2024-05-04T12:19:23.650152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:05:34.635703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.808268+0000","last_change":"2024-05-04T12:19:22.677444+0000","last_active":"2024-05-04T12:19:23.808268+0000","last_peered":"2024-05-04T12:19:23.808268+0000","last_clean":"2024-05-04T12:19:23.808268+0000","last_became_active":"2024-05-04T12:19:22.677192+0000","last_became_peered":"2024-05-04T12:19:22.677192+0000","last_unstale":"2024-05-04T12:19:23.808268+0000","last_undegraded":"2024-05-04T12:19:23.808268+0000","last_fullsized":"2024-05-04T12:19:23.808268+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:20:08.590647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.808852+0000","last_change":"2024-05-04T12:19:21.663720+0000","last_active":"2024-05-04T12:19:23.808852+0000","last_peered":"2024-05-04T12:19:23.808852+0000","last_clean":"2024-05-04T12:19:23.808852+0000","last_became_active":"2024-05-04T12:19:21.661342+0000","last_became_peered":"2024-05-04T12:19:21.661342+0000","last_unstale":"2024-05-04T12:19:23.808852+0000","last_undegraded":"2024-05-04T12:19:23.808852+0000","last_fullsized":"2024-05-04T12:19:23.808852+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:05:34.635703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.648708+0000","last_change":"2024-05-04T12:19:22.696510+0000","last_active":"2024-05-04T12:19:23.648708+0000","last_peered":"2024-05-04T12:19:23.648708+0000","last_clean":"2024-05-04T12:19:23.648708+0000","last_became_active":"2024-05-04T12:19:22.695751+0000","last_became_peered":"2024-05-04T12:19:22.695751+0000","last_unstale":"2024-05-04T12:19:23.648708+0000","last_undegraded":"2024-05-04T12:19:23.648708+0000","last_fullsized":"2024-05-04T12:19:23.648708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:49:28.269102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.810202+0000","last_change":"2024-05-04T12:19:21.672395+0000","last_active":"2024-05-04T12:19:23.810202+0000","last_peered":"2024-05-04T12:19:23.810202+0000","last_clean":"2024-05-04T12:19:23.810202+0000","last_became_active":"2024-05-04T12:19:21.671600+0000","last_became_peered":"2024-05-04T12:19:21.671600+0000","last_unstale":"2024-05-04T12:19:23.810202+0000","last_undegraded":"2024-05-04T12:19:23.810202+0000","last_fullsized":"2024-05-04T12:19:23.810202+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T20:42:38.052914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.808808+0000","last_change":"2024-05-04T12:19:22.666640+0000","last_active":"2024-05-04T12:19:23.808808+0000","last_peered":"2024-05-04T12:19:23.808808+0000","last_clean":"2024-05-04T12:19:23.808808+0000","last_became_active":"2024-05-04T12:19:22.665931+0000","last_became_peered":"2024-05-04T12:19:22.665931+0000","last_unstale":"2024-05-04T12:19:23.808808+0000","last_undegraded":"2024-05-04T12:19:23.808808+0000","last_fullsized":"2024-05-04T12:19:23.808808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:43:25.107887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.688037+0000","last_change":"2024-05-04T12:19:21.673552+0000","last_active":"2024-05-04T12:19:23.688037+0000","last_peered":"2024-05-04T12:19:23.688037+0000","last_clean":"2024-05-04T12:19:23.688037+0000","last_became_active":"2024-05-04T12:19:21.673207+0000","last_became_peered":"2024-05-04T12:19:21.673207+0000","last_unstale":"2024-05-04T12:19:23.688037+0000","last_undegraded":"2024-05-04T12:19:23.688037+0000","last_fullsized":"2024-05-04T12:19:23.688037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T23:12:22.175099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.812179+0000","last_change":"2024-05-04T12:19:22.688001+0000","last_active":"2024-05-04T12:19:23.812179+0000","last_peered":"2024-05-04T12:19:23.812179+0000","last_clean":"2024-05-04T12:19:23.812179+0000","last_became_active":"2024-05-04T12:19:22.687014+0000","last_became_peered":"2024-05-04T12:19:22.687014+0000","last_unstale":"2024-05-04T12:19:23.812179+0000","last_undegraded":"2024-05-04T12:19:23.812179+0000","last_fullsized":"2024-05-04T12:19:23.812179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T19:57:51.506509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.686140+0000","last_change":"2024-05-04T12:19:21.659944+0000","last_active":"2024-05-04T12:19:23.686140+0000","last_peered":"2024-05-04T12:19:23.686140+0000","last_clean":"2024-05-04T12:19:23.686140+0000","last_became_active":"2024-05-04T12:19:21.659644+0000","last_became_peered":"2024-05-04T12:19:21.659644+0000","last_unstale":"2024-05-04T12:19:23.686140+0000","last_undegraded":"2024-05-04T12:19:23.686140+0000","last_fullsized":"2024-05-04T12:19:23.686140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T17:36:37.379146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.684028+0000","last_change":"2024-05-04T12:19:22.674929+0000","last_active":"2024-05-04T12:19:23.684028+0000","last_peered":"2024-05-04T12:19:23.684028+0000","last_clean":"2024-05-04T12:19:23.684028+0000","last_became_active":"2024-05-04T12:19:22.674537+0000","last_became_peered":"2024-05-04T12:19:22.674537+0000","last_unstale":"2024-05-04T12:19:23.684028+0000","last_undegraded":"2024-05-04T12:19:23.684028+0000","last_fullsized":"2024-05-04T12:19:23.684028+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:20:05.965288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.687569+0000","last_change":"2024-05-04T12:19:21.656136+0000","last_active":"2024-05-04T12:19:23.687569+0000","last_peered":"2024-05-04T12:19:23.687569+0000","last_clean":"2024-05-04T12:19:23.687569+0000","last_became_active":"2024-05-04T12:19:21.655722+0000","last_became_peered":"2024-05-04T12:19:21.655722+0000","last_unstale":"2024-05-04T12:19:23.687569+0000","last_undegraded":"2024-05-04T12:19:23.687569+0000","last_fullsized":"2024-05-04T12:19:23.687569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:36:50.612240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.647861+0000","last_change":"2024-05-04T12:19:22.679522+0000","last_active":"2024-05-04T12:19:23.647861+0000","last_peered":"2024-05-04T12:19:23.647861+0000","last_clean":"2024-05-04T12:19:23.647861+0000","last_became_active":"2024-05-04T12:19:22.678784+0000","last_became_peered":"2024-05-04T12:19:22.678784+0000","last_unstale":"2024-05-04T12:19:23.647861+0000","last_undegraded":"2024-05-04T12:19:23.647861+0000","last_fullsized":"2024-05-04T12:19:23.647861+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T22:47:33.717831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.684728+0000","last_change":"2024-05-04T12:19:21.665600+0000","last_active":"2024-05-04T12:19:23.684728+0000","last_peered":"2024-05-04T12:19:23.684728+0000","last_clean":"2024-05-04T12:19:23.684728+0000","last_became_active":"2024-05-04T12:19:21.665259+0000","last_became_peered":"2024-05-04T12:19:21.665259+0000","last_unstale":"2024-05-04T12:19:23.684728+0000","last_undegraded":"2024-05-04T12:19:23.684728+0000","last_fullsized":"2024-05-04T12:19:23.684728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:43:24.099486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.684740+0000","last_change":"2024-05-04T12:19:22.671129+0000","last_active":"2024-05-04T12:19:23.684740+0000","last_peered":"2024-05-04T12:19:23.684740+0000","last_clean":"2024-05-04T12:19:23.684740+0000","last_became_active":"2024-05-04T12:19:22.670040+0000","last_became_peered":"2024-05-04T12:19:22.670040+0000","last_unstale":"2024-05-04T12:19:23.684740+0000","last_undegraded":"2024-05-04T12:19:23.684740+0000","last_fullsized":"2024-05-04T12:19:23.684740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T17:52:39.897158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.687290+0000","last_change":"2024-05-04T12:19:21.673336+0000","last_active":"2024-05-04T12:19:23.687290+0000","last_peered":"2024-05-04T12:19:23.687290+0000","last_clean":"2024-05-04T12:19:23.687290+0000","last_became_active":"2024-05-04T12:19:21.673056+0000","last_became_peered":"2024-05-04T12:19:21.673056+0000","last_unstale":"2024-05-04T12:19:23.687290+0000","last_undegraded":"2024-05-04T12:19:23.687290+0000","last_fullsized":"2024-05-04T12:19:23.687290+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T21:30:49.875995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.645302+0000","last_change":"2024-05-04T12:19:22.678482+0000","last_active":"2024-05-04T12:19:23.645302+0000","last_peered":"2024-05-04T12:19:23.645302+0000","last_clean":"2024-05-04T12:19:23.645302+0000","last_became_active":"2024-05-04T12:19:22.677838+0000","last_became_peered":"2024-05-04T12:19:22.677838+0000","last_unstale":"2024-05-04T12:19:23.645302+0000","last_undegraded":"2024-05-04T12:19:23.645302+0000","last_fullsized":"2024-05-04T12:19:23.645302+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T17:32:05.053202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.809106+0000","last_change":"2024-05-04T12:19:21.672752+0000","last_active":"2024-05-04T12:19:23.809106+0000","last_peered":"2024-05-04T12:19:23.809106+0000","last_clean":"2024-05-04T12:19:23.809106+0000","last_became_active":"2024-05-04T12:19:21.672039+0000","last_became_peered":"2024-05-04T12:19:21.672039+0000","last_unstale":"2024-05-04T12:19:23.809106+0000","last_undegraded":"2024-05-04T12:19:23.809106+0000","last_fullsized":"2024-05-04T12:19:23.809106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T22:32:16.151758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.808813+0000","last_change":"2024-05-04T12:19:22.666032+0000","last_active":"2024-05-04T12:19:23.808813+0000","last_peered":"2024-05-04T12:19:23.808813+0000","last_clean":"2024-05-04T12:19:23.808813+0000","last_became_active":"2024-05-04T12:19:22.665244+0000","last_became_peered":"2024-05-04T12:19:22.665244+0000","last_unstale":"2024-05-04T12:19:23.808813+0000","last_undegraded":"2024-05-04T12:19:23.808813+0000","last_fullsized":"2024-05-04T12:19:23.808813+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T22:27:24.827385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.645105+0000","last_change":"2024-05-04T12:19:21.667231+0000","last_active":"2024-05-04T12:19:23.645105+0000","last_peered":"2024-05-04T12:19:23.645105+0000","last_clean":"2024-05-04T12:19:23.645105+0000","last_became_active":"2024-05-04T12:19:21.666870+0000","last_became_peered":"2024-05-04T12:19:21.666870+0000","last_unstale":"2024-05-04T12:19:23.645105+0000","last_undegraded":"2024-05-04T12:19:23.645105+0000","last_fullsized":"2024-05-04T12:19:23.645105+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T16:39:58.703614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.649970+0000","last_change":"2024-05-04T12:19:22.696753+0000","last_active":"2024-05-04T12:19:23.649970+0000","last_peered":"2024-05-04T12:19:23.649970+0000","last_clean":"2024-05-04T12:19:23.649970+0000","last_became_active":"2024-05-04T12:19:22.695971+0000","last_became_peered":"2024-05-04T12:19:22.695971+0000","last_unstale":"2024-05-04T12:19:23.649970+0000","last_undegraded":"2024-05-04T12:19:23.649970+0000","last_fullsized":"2024-05-04T12:19:23.649970+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:21.625858+0000","last_clean_scrub_stamp":"2024-05-04T12:19:21.625858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:13:51.860759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.649884+0000","last_change":"2024-05-04T12:19:21.681560+0000","last_active":"2024-05-04T12:19:23.649884+0000","last_peered":"2024-05-04T12:19:23.649884+0000","last_clean":"2024-05-04T12:19:23.649884+0000","last_became_active":"2024-05-04T12:19:21.680784+0000","last_became_peered":"2024-05-04T12:19:21.680784+0000","last_unstale":"2024-05-04T12:19:23.649884+0000","last_undegraded":"2024-05-04T12:19:23.649884+0000","last_fullsized":"2024-05-04T12:19:23.649884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:19:20.617457+0000","last_clean_scrub_stamp":"2024-05-04T12:19:20.617457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T22:32:16.151758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-04T12:19:23.649365+0000","last_change":"2024-05-04T12:18:12.644577+0000","last_active":"2024-05-04T12:19:23.649365+0000","last_peered":"2024-05-04T12:19:23.649365+0000","last_clean":"2024-05-04T12:19:23.649365+0000","last_became_active":"2024-05-04T12:18:12.640041+0000","last_became_peered":"2024-05-04T12:18:12.640041+0000","last_unstale":"2024-05-04T12:19:23.649365+0000","last_undegraded":"2024-05-04T12:19:23.649365+0000","last_fullsized":"2024-05-04T12:19:23.649365+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_clean_scrub_stamp":"2024-05-04T12:18:11.260157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:15:02.089387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":741907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":152065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":152121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":151553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":153083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":151519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":153083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":978944,"data_stored":741817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T12:19:27.090 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-04T12:19:27.090 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-04T12:19:27.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe87de0d640 1 Processor -- start 2024-05-04T12:19:27.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe87de0d640 1 -- start start 2024-05-04T12:19:27.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe87de0d640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe87815c590 0x7fe8780cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:27.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe87de0d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe8780cccd0 0x7fe87806e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:27.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe87de0d640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe87806dbf0 con 0x7fe878145960 2024-05-04T12:19:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe87de0d640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fe87806cfe0 con 0x7fe8780cccd0 2024-05-04T12:19:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe87de0d640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe87806d2c0 con 0x7fe87815c590 2024-05-04T12:19:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe877fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe8780cccd0 0x7fe87806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe8777fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fe878145960 0x7fe878160590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:41112/0 (socket says 172.21.15.52:41112) 2024-05-04T12:19:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe8777fe640 1 -- 172.21.15.52:0/3253685729 learned_addr learned my addr 172.21.15.52:0/3253685729 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe877fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe8780cccd0 0x7fe87806e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59590/0 (socket says 172.21.15.52:59590) 2024-05-04T12:19:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe876ffd640 1 --2- 172.21.15.52:0/3253685729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe87815c590 0x7fe8780cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe8767fc640 1 -- 172.21.15.52:0/3253685729 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 680259477 0 0) 0x7fe87806dbf0 con 0x7fe878145960 2024-05-04T12:19:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe8767fc640 1 -- 172.21.15.52:0/3253685729 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe860003530 con 0x7fe878145960 2024-05-04T12:19:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe877fff640 1 -- 172.21.15.52:0/3253685729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe87815c590 msgr2=0x7fe8780cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe877fff640 1 --2- 172.21.15.52:0/3253685729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe87815c590 0x7fe8780cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe877fff640 1 -- 172.21.15.52:0/3253685729 >> v1:172.21.15.52:6789/0 conn(0x7fe878145960 legacy=0x7fe878160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.265+0000 7fe877fff640 1 -- 172.21.15.52:0/3253685729 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87806d5a0 con 0x7fe8780cccd0 2024-05-04T12:19:27.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.269+0000 7fe877fff640 1 --2- 172.21.15.52:0/3253685729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8780cccd0 0x7fe87806e080 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe8680097c0 tx=0x7fe86801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afe22849920c942 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:27.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.269+0000 7fe8767fc640 1 -- 172.21.15.52:0/3253685729 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8680b0070 con 0x7fe8780cccd0 2024-05-04T12:19:27.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.269+0000 7fe8767fc640 1 -- 172.21.15.52:0/3253685729 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8680ab600 con 0x7fe8780cccd0 2024-05-04T12:19:27.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.269+0000 7fe8767fc640 1 -- 172.21.15.52:0/3253685729 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8680ab9d0 con 0x7fe8780cccd0 2024-05-04T12:19:27.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.269+0000 7fe87de0d640 1 -- 172.21.15.52:0/3253685729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8780cccd0 msgr2=0x7fe87806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.269+0000 7fe87de0d640 1 --2- 172.21.15.52:0/3253685729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8780cccd0 0x7fe87806e080 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe8680097c0 tx=0x7fe86801c410 comp rx=0 tx=0).stop 2024-05-04T12:19:27.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.269+0000 7fe87de0d640 1 -- 172.21.15.52:0/3253685729 shutdown_connections 2024-05-04T12:19:27.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.269+0000 7fe87de0d640 1 --2- 172.21.15.52:0/3253685729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8780cccd0 0x7fe87806e080 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.269+0000 7fe87de0d640 1 --2- 172.21.15.52:0/3253685729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe87815c590 0x7fe8780cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.269+0000 7fe87de0d640 1 -- 172.21.15.52:0/3253685729 >> 172.21.15.52:0/3253685729 conn(0x7fe878067730 msgr2=0x7fe8780b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:27.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.269+0000 7fe87de0d640 1 -- 172.21.15.52:0/3253685729 shutdown_connections 2024-05-04T12:19:27.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.269+0000 7fe87de0d640 1 -- 172.21.15.52:0/3253685729 wait complete. 2024-05-04T12:19:27.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.269+0000 7fe87de0d640 1 Processor -- start 2024-05-04T12:19:27.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.269+0000 7fe87de0d640 1 -- start start 2024-05-04T12:19:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe87de0d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8780cccd0 0x7fe878132bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe87de0d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe878145960 0x7fe87815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe87de0d640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe87815c590 0x7fe8780adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe87de0d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe878145ec0 con 0x7fe8780cccd0 2024-05-04T12:19:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe87de0d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe878145d40 con 0x7fe878145960 2024-05-04T12:19:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe87de0d640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe87806edd0 con 0x7fe87815c590 2024-05-04T12:19:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe8777fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8780cccd0 0x7fe878132bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe8777fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8780cccd0 0x7fe878132bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59598/0 (socket says 172.21.15.52:59598) 2024-05-04T12:19:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe8777fe640 1 -- 172.21.15.52:0/2777102907 learned_addr learned my addr 172.21.15.52:0/2777102907 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe877fff640 1 --2- 172.21.15.52:0/2777102907 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe87815c590 0x7fe8780adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe8777fe640 1 -- 172.21.15.52:0/2777102907 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe87815c590 msgr2=0x7fe8780adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe8777fe640 1 --2- 172.21.15.52:0/2777102907 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe87815c590 0x7fe8780adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe8777fe640 1 -- 172.21.15.52:0/2777102907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe878145960 msgr2=0x7fe87815a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe8777fe640 1 --2- 172.21.15.52:0/2777102907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe878145960 0x7fe87815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe8777fe640 1 -- 172.21.15.52:0/2777102907 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe878133980 con 0x7fe8780cccd0 2024-05-04T12:19:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe8777fe640 1 --2- 172.21.15.52:0/2777102907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8780cccd0 0x7fe878132bb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe87000cb40 tx=0x7fe87000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe85ffff640 1 -- 172.21.15.52:0/2777102907 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870017070 con 0x7fe8780cccd0 2024-05-04T12:19:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe85ffff640 1 -- 172.21.15.52:0/2777102907 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87000aca0 con 0x7fe8780cccd0 2024-05-04T12:19:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe87de0d640 1 -- 172.21.15.52:0/2777102907 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe878133c60 con 0x7fe8780cccd0 2024-05-04T12:19:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe85ffff640 1 -- 172.21.15.52:0/2777102907 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870012620 con 0x7fe8780cccd0 2024-05-04T12:19:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe87de0d640 1 -- 172.21.15.52:0/2777102907 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe878134220 con 0x7fe8780cccd0 2024-05-04T12:19:27.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe87de0d640 1 -- 172.21.15.52:0/2777102907 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe878062e50 con 0x7fe8780cccd0 2024-05-04T12:19:27.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe85ffff640 1 -- 172.21.15.52:0/2777102907 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe87001a070 con 0x7fe8780cccd0 2024-05-04T12:19:27.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe85ffff640 1 --2- 172.21.15.52:0/2777102907 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe8640678c0 0x7fe864069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:27.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe85ffff640 1 -- 172.21.15.52:0/2777102907 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe870085af0 con 0x7fe8780cccd0 2024-05-04T12:19:27.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.273+0000 7fe876ffd640 1 --2- 172.21.15.52:0/2777102907 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe8640678c0 0x7fe864069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:27.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.277+0000 7fe876ffd640 1 --2- 172.21.15.52:0/2777102907 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe8640678c0 0x7fe864069d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe86c006aa0 tx=0x7fe86c006a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:27.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.281+0000 7fe85ffff640 1 -- 172.21.15.52:0/2777102907 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe870056630 con 0x7fe8780cccd0 2024-05-04T12:19:27.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.501+0000 7fe87de0d640 1 -- 172.21.15.52:0/2777102907 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe878075320 con 0x7fe8780cccd0 2024-05-04T12:19:27.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.505+0000 7fe85ffff640 1 -- 172.21.15.52:0/2777102907 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fe870055f30 con 0x7fe8780cccd0 2024-05-04T12:19:27.508 INFO:teuthology.orchestra.run.smithi052.stdout:[client.0] 2024-05-04T12:19:27.508 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBVJzZm/x1tLhAApncXD+0NjD0NbekN+kVgdQ== 2024-05-04T12:19:27.508 INFO:teuthology.orchestra.run.smithi052.stdout: caps mds = "allow" 2024-05-04T12:19:27.508 INFO:teuthology.orchestra.run.smithi052.stdout: caps mon = "allow r" 2024-05-04T12:19:27.508 INFO:teuthology.orchestra.run.smithi052.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-04T12:19:27.509 INFO:teuthology.orchestra.run.smithi052.stderr:updated caps for client.0 2024-05-04T12:19:27.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.505+0000 7fe87de0d640 1 -- 172.21.15.52:0/2777102907 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe8640678c0 msgr2=0x7fe864069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.505+0000 7fe87de0d640 1 --2- 172.21.15.52:0/2777102907 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe8640678c0 0x7fe864069d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe86c006aa0 tx=0x7fe86c006a30 comp rx=0 tx=0).stop 2024-05-04T12:19:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.505+0000 7fe87de0d640 1 -- 172.21.15.52:0/2777102907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8780cccd0 msgr2=0x7fe878132bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.505+0000 7fe87de0d640 1 --2- 172.21.15.52:0/2777102907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8780cccd0 0x7fe878132bb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe87000cb40 tx=0x7fe87000a060 comp rx=0 tx=0).stop 2024-05-04T12:19:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.505+0000 7fe87de0d640 1 -- 172.21.15.52:0/2777102907 shutdown_connections 2024-05-04T12:19:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.505+0000 7fe87de0d640 1 --2- 172.21.15.52:0/2777102907 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe8640678c0 0x7fe864069d80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.505+0000 7fe87de0d640 1 --2- 172.21.15.52:0/2777102907 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe87815c590 0x7fe8780adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.505+0000 7fe87de0d640 1 --2- 172.21.15.52:0/2777102907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe878145960 0x7fe87815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.505+0000 7fe87de0d640 1 --2- 172.21.15.52:0/2777102907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8780cccd0 0x7fe878132bb0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.505+0000 7fe87de0d640 1 -- 172.21.15.52:0/2777102907 >> 172.21.15.52:0/2777102907 conn(0x7fe878067730 msgr2=0x7fe87813abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.505+0000 7fe87de0d640 1 -- 172.21.15.52:0/2777102907 shutdown_connections 2024-05-04T12:19:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.505+0000 7fe87de0d640 1 -- 172.21.15.52:0/2777102907 wait complete. 2024-05-04T12:19:27.528 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T12:19:27.616 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T12:19:27.613+0000 7fbf8de13640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-04T12:19:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.645+0000 7f6ffa113640 1 Processor -- start 2024-05-04T12:19:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.645+0000 7f6ffa113640 1 -- start start 2024-05-04T12:19:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.645+0000 7f6ffa113640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ff406db40 0x7f6ff4067070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.645+0000 7f6ffa113640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ff406f720 con 0x7f6ff415ae50 2024-05-04T12:19:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.645+0000 7f6ffa113640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ff40cea70 con 0x7f6ff406f290 2024-05-04T12:19:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.645+0000 7f6ffa113640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff4067770 con 0x7f6ff406db40 2024-05-04T12:19:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff37fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f6ff415ae50 0x7f6ff406d430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:41118/0 (socket says 172.21.15.52:41118) 2024-05-04T12:19:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff37fe640 1 -- 172.21.15.52:0/2345920245 learned_addr learned my addr 172.21.15.52:0/2345920245 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:27.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff2ffd640 1 --2- 172.21.15.52:0/2345920245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ff406db40 0x7f6ff4067070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:27.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 -- 172.21.15.52:0/2345920245 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1769990909 0 0) 0x7f6ff406f720 con 0x7f6ff415ae50 2024-05-04T12:19:27.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 -- 172.21.15.52:0/2345920245 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fe0003530 con 0x7f6ff415ae50 2024-05-04T12:19:27.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 -- 172.21.15.52:0/2345920245 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 799361567 0 0) 0x7f6fe0003530 con 0x7f6ff415ae50 2024-05-04T12:19:27.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 -- 172.21.15.52:0/2345920245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ff406db40 msgr2=0x7f6ff4067070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 --2- 172.21.15.52:0/2345920245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ff406db40 0x7f6ff4067070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 -- 172.21.15.52:0/2345920245 >> v1:172.21.15.111:6789/0 conn(0x7f6ff406f290 legacy=0x7f6ff40cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 -- 172.21.15.52:0/2345920245 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff40ced10 con 0x7f6ff415ae50 2024-05-04T12:19:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 -- 172.21.15.52:0/2345920245 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f6fe4001a00 con 0x7f6ff415ae50 2024-05-04T12:19:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 -- 172.21.15.52:0/2345920245 >> v1:172.21.15.52:6789/0 conn(0x7f6ff415ae50 legacy=0x7f6ff406d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 --2- 172.21.15.52:0/2345920245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fe00039f0 0x7f6fe0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 --2- 172.21.15.52:0/2345920245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ff406db40 0x7f6fe0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 --2- 172.21.15.52:0/2345920245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fe0004d50 0x7f6fe00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 -- 172.21.15.52:0/2345920245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff4067770 con 0x7f6fe00039f0 2024-05-04T12:19:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 -- 172.21.15.52:0/2345920245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe000ba80 con 0x7f6fe0004d50 2024-05-04T12:19:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff27fc640 1 -- 172.21.15.52:0/2345920245 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f6fe000de40 con 0x7f6ff406db40 2024-05-04T12:19:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff37fe640 1 --2- 172.21.15.52:0/2345920245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fe00039f0 0x7f6fe0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff2ffd640 1 --2- 172.21.15.52:0/2345920245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ff406db40 0x7f6fe0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff3fff640 1 --2- 172.21.15.52:0/2345920245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fe0004d50 0x7f6fe00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff37fe640 1 -- 172.21.15.52:0/2345920245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ff406db40 msgr2=0x7f6fe0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff37fe640 1 --2- 172.21.15.52:0/2345920245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ff406db40 0x7f6fe0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff37fe640 1 -- 172.21.15.52:0/2345920245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fe0004d50 msgr2=0x7f6fe00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff37fe640 1 --2- 172.21.15.52:0/2345920245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fe0004d50 0x7f6fe00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.649+0000 7f6ff37fe640 1 -- 172.21.15.52:0/2345920245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe0003530 con 0x7f6fe00039f0 2024-05-04T12:19:27.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff37fe640 1 --2- 172.21.15.52:0/2345920245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fe00039f0 0x7f6fe0003dd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400a7b0 tx=0x7f6fe401d2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea57460e664ab7f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:27.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff27fc640 1 -- 172.21.15.52:0/2345920245 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe400a3d0 con 0x7f6fe00039f0 2024-05-04T12:19:27.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff27fc640 1 -- 172.21.15.52:0/2345920245 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe40a5e50 con 0x7f6fe00039f0 2024-05-04T12:19:27.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff27fc640 1 -- 172.21.15.52:0/2345920245 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe40ad870 con 0x7f6fe00039f0 2024-05-04T12:19:27.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ffa113640 1 -- 172.21.15.52:0/2345920245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fe00039f0 msgr2=0x7f6fe0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ffa113640 1 --2- 172.21.15.52:0/2345920245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fe00039f0 0x7f6fe0003dd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400a7b0 tx=0x7f6fe401d2b0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff3fff640 1 -- 172.21.15.52:0/2345920245 reap_dead start 2024-05-04T12:19:27.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ffa113640 1 -- 172.21.15.52:0/2345920245 shutdown_connections 2024-05-04T12:19:27.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ffa113640 1 -- 172.21.15.52:0/2345920245 >> 172.21.15.52:0/2345920245 conn(0x7f6ff40b3190 msgr2=0x7f6ff40b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:27.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ffa113640 1 -- 172.21.15.52:0/2345920245 shutdown_connections 2024-05-04T12:19:27.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ffa113640 1 -- 172.21.15.52:0/2345920245 wait complete. 2024-05-04T12:19:27.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ffa113640 1 Processor -- start 2024-05-04T12:19:27.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ffa113640 1 -- start start 2024-05-04T12:19:27.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ffa113640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6fe00039f0 0x7f6ff4145490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:27.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ffa113640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ff40cdcf0 0x7f6ff414c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:27.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ffa113640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ff40ce170 0x7f6ff414cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:27.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ffa113640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff415b230 con 0x7f6ff40ce170 2024-05-04T12:19:27.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ffa113640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff4071d80 con 0x7f6ff40cdcf0 2024-05-04T12:19:27.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ffa113640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff415b3b0 con 0x7f6fe00039f0 2024-05-04T12:19:27.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ff40ce170 0x7f6ff414cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:27.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ff40ce170 0x7f6ff414cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59610/0 (socket says 172.21.15.52:59610) 2024-05-04T12:19:27.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff3fff640 1 -- 172.21.15.52:0/596654789 learned_addr learned my addr 172.21.15.52:0/596654789 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:27.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff2ffd640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ff40cdcf0 0x7f6ff414c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:27.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff37fe640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6fe00039f0 0x7f6ff4145490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:27.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff3fff640 1 -- 172.21.15.52:0/596654789 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6fe00039f0 msgr2=0x7f6ff4145490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff3fff640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6fe00039f0 0x7f6ff4145490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff3fff640 1 -- 172.21.15.52:0/596654789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ff40cdcf0 msgr2=0x7f6ff414c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff3fff640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ff40cdcf0 0x7f6ff414c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff3fff640 1 -- 172.21.15.52:0/596654789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff414a6b0 con 0x7f6ff40ce170 2024-05-04T12:19:27.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff37fe640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6fe00039f0 0x7f6ff4145490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:27.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6ff3fff640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ff40ce170 0x7f6ff414cd20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40ccd60 tx=0x7f6fec00e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:27.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6febfff640 1 -- 172.21.15.52:0/596654789 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec019070 con 0x7f6ff40ce170 2024-05-04T12:19:27.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6febfff640 1 -- 172.21.15.52:0/596654789 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fec0052e0 con 0x7f6ff40ce170 2024-05-04T12:19:27.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.653+0000 7f6febfff640 1 -- 172.21.15.52:0/596654789 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec003840 con 0x7f6ff40ce170 2024-05-04T12:19:27.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.657+0000 7f6ffa113640 1 -- 172.21.15.52:0/596654789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff414a990 con 0x7f6ff40ce170 2024-05-04T12:19:27.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.657+0000 7f6ffa113640 1 -- 172.21.15.52:0/596654789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff414af50 con 0x7f6ff40ce170 2024-05-04T12:19:27.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.657+0000 7f6ffa113640 1 -- 172.21.15.52:0/596654789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff4062e50 con 0x7f6ff40ce170 2024-05-04T12:19:27.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.657+0000 7f6febfff640 1 -- 172.21.15.52:0/596654789 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6fec01c070 con 0x7f6ff40ce170 2024-05-04T12:19:27.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.657+0000 7f6febfff640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6fd40678c0 0x7f6fd4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:27.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.657+0000 7f6febfff640 1 -- 172.21.15.52:0/596654789 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f6fec085190 con 0x7f6ff40ce170 2024-05-04T12:19:27.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.657+0000 7f6ff37fe640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6fd40678c0 0x7f6fd4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:27.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.657+0000 7f6ff37fe640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6fd40678c0 0x7f6fd4069d80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40ad000 tx=0x7f6fe40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.661+0000 7f6febfff640 1 -- 172.21.15.52:0/596654789 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6fec055cd0 con 0x7f6ff40ce170 2024-05-04T12:19:27.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.861+0000 7f6ffa113640 1 -- 172.21.15.52:0/596654789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ff40ccb40 con 0x7f6ff40ce170 2024-05-04T12:19:27.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.861+0000 7f6febfff640 1 -- 172.21.15.52:0/596654789 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5184 (secure 0 0 0) 0x7f6fec0555d0 con 0x7f6ff40ce170 2024-05-04T12:19:27.865 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:19:27.865 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2438654614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2438654614},{"type":"v1","addr":"172.21.15.111:6835","nonce":2438654614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/796226117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":796226117},{"type":"v1","addr":"172.21.15.52:6835","nonce":796226117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3866938727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3866938727},{"type":"v1","addr":"172.21.15.111:6833","nonce":3866938727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-04T12:19:22.017880+0000","modified":"2024-05-04T12:19:25.052316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5199":{"gid":5199,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6837/1620779861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1620779861},{"type":"v1","addr":"172.21.15.52:6837","nonce":1620779861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5199,"qdb_cluster":[5199]},"id":2}]} 2024-05-04T12:19:27.866 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 16 2024-05-04T12:19:27.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.865+0000 7f6ffa113640 1 -- 172.21.15.52:0/596654789 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6fd40678c0 msgr2=0x7f6fd4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.865+0000 7f6ffa113640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6fd40678c0 0x7f6fd4069d80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40ad000 tx=0x7f6fe40a7000 comp rx=0 tx=0).stop 2024-05-04T12:19:27.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.865+0000 7f6ffa113640 1 -- 172.21.15.52:0/596654789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ff40ce170 msgr2=0x7f6ff414cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:27.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.865+0000 7f6ffa113640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ff40ce170 0x7f6ff414cd20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40ccd60 tx=0x7f6fec00e820 comp rx=0 tx=0).stop 2024-05-04T12:19:27.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.865+0000 7f6ffa113640 1 -- 172.21.15.52:0/596654789 shutdown_connections 2024-05-04T12:19:27.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.865+0000 7f6ffa113640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6fd40678c0 0x7f6fd4069d80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.865+0000 7f6ffa113640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ff40ce170 0x7f6ff414cd20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.865+0000 7f6ffa113640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ff40cdcf0 0x7f6ff414c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.865+0000 7f6ffa113640 1 --2- 172.21.15.52:0/596654789 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6fe00039f0 0x7f6ff4145490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:27.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.865+0000 7f6ffa113640 1 -- 172.21.15.52:0/596654789 >> 172.21.15.52:0/596654789 conn(0x7f6ff40b3190 msgr2=0x7f6ff4073200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:27.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.865+0000 7f6ffa113640 1 -- 172.21.15.52:0/596654789 shutdown_connections 2024-05-04T12:19:27.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:27.865+0000 7f6ffa113640 1 -- 172.21.15.52:0/596654789 wait complete. 2024-05-04T12:19:28.286 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-04T12:19:22.017880+0000', 'modified': '2024-05-04T12:19:25.052316+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5199}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5199': {'gid': 5199, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.52:6837/1620779861', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6836', 'nonce': 1620779861}, {'type': 'v1', 'addr': '172.21.15.52:6837', 'nonce': 1620779861}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5199, 'qdb_cluster': [5199]} 2024-05-04T12:19:28.287 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-04T12:19:28.287 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-04T12:19:28.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.417+0000 7f38a9316640 1 Processor -- start 2024-05-04T12:19:28.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.417+0000 7f38a9316640 1 -- start start 2024-05-04T12:19:28.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.417+0000 7f38a9316640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f38a415bf50 0x7f38a415c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.417+0000 7f38a9316640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f38a40aa0c0 0x7f38a40ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.417+0000 7f38a9316640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38a406d1b0 con 0x7f38a40aa550 2024-05-04T12:19:28.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.417+0000 7f38a9316640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f38a406d490 con 0x7f38a415bf50 2024-05-04T12:19:28.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.417+0000 7f38a9316640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f38a406d770 con 0x7f38a40aa0c0 2024-05-04T12:19:28.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.417+0000 7f38a2ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f38a415bf50 0x7f38a415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.417+0000 7f38a37fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f38a40aa550 0x7f38a406e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:41120/0 (socket says 172.21.15.52:41120) 2024-05-04T12:19:28.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.417+0000 7f38a37fe640 1 -- 172.21.15.52:0/4078285741 learned_addr learned my addr 172.21.15.52:0/4078285741 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T12:19:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.417+0000 7f389bfff640 1 --2- 172.21.15.52:0/4078285741 >> v2:172.21.15.111:3300/0 conn(0x7f38a40aa0c0 0x7f38a40ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a27fc640 1 -- 172.21.15.52:0/4078285741 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4109576322 0 0) 0x7f38a406d1b0 con 0x7f38a40aa550 2024-05-04T12:19:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a27fc640 1 -- 172.21.15.52:0/4078285741 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3890003530 con 0x7f38a40aa550 2024-05-04T12:19:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a2ffd640 1 -- 172.21.15.52:0/4078285741 >> v2:172.21.15.111:3300/0 conn(0x7f38a40aa0c0 msgr2=0x7f38a40ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a2ffd640 1 --2- 172.21.15.52:0/4078285741 >> v2:172.21.15.111:3300/0 conn(0x7f38a40aa0c0 0x7f38a40ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a2ffd640 1 -- 172.21.15.52:0/4078285741 >> v1:172.21.15.52:6789/0 conn(0x7f38a40aa550 legacy=0x7f38a406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a2ffd640 1 -- 172.21.15.52:0/4078285741 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a406da50 con 0x7f38a415bf50 2024-05-04T12:19:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a2ffd640 1 --2- 172.21.15.52:0/4078285741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38a415bf50 0x7f38a415c330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f38a40b3ae0 tx=0x7f389401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f2a8c95885aa07 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a27fc640 1 -- 172.21.15.52:0/4078285741 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38940a7070 con 0x7f38a415bf50 2024-05-04T12:19:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a27fc640 1 -- 172.21.15.52:0/4078285741 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38940b0070 con 0x7f38a415bf50 2024-05-04T12:19:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a27fc640 1 -- 172.21.15.52:0/4078285741 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38940ab890 con 0x7f38a415bf50 2024-05-04T12:19:28.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a9316640 1 -- 172.21.15.52:0/4078285741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38a415bf50 msgr2=0x7f38a415c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a9316640 1 --2- 172.21.15.52:0/4078285741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38a415bf50 0x7f38a415c330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f38a40b3ae0 tx=0x7f389401c450 comp rx=0 tx=0).stop 2024-05-04T12:19:28.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a9316640 1 -- 172.21.15.52:0/4078285741 shutdown_connections 2024-05-04T12:19:28.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a9316640 1 --2- 172.21.15.52:0/4078285741 >> v2:172.21.15.111:3300/0 conn(0x7f38a40aa0c0 0x7f38a40ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a9316640 1 --2- 172.21.15.52:0/4078285741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38a415bf50 0x7f38a415c330 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a9316640 1 -- 172.21.15.52:0/4078285741 >> 172.21.15.52:0/4078285741 conn(0x7f38a40b14a0 msgr2=0x7f38a40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:28.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a9316640 1 -- 172.21.15.52:0/4078285741 shutdown_connections 2024-05-04T12:19:28.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.421+0000 7f38a9316640 1 -- 172.21.15.52:0/4078285741 wait complete. 2024-05-04T12:19:28.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a9316640 1 Processor -- start 2024-05-04T12:19:28.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a9316640 1 -- start start 2024-05-04T12:19:28.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a9316640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38a40aa0c0 0x7f38a4152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a9316640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38a40aa550 0x7f38a4150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a9316640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38a415bf50 0x7f38a4150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a9316640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f38a4169030 con 0x7f38a40aa0c0 2024-05-04T12:19:28.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a9316640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f38a40cd4a0 con 0x7f38a40aa550 2024-05-04T12:19:28.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a9316640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f38a41691b0 con 0x7f38a415bf50 2024-05-04T12:19:28.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38a40aa0c0 0x7f38a4152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38a40aa0c0 0x7f38a4152d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59624/0 (socket says 172.21.15.52:59624) 2024-05-04T12:19:28.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a2ffd640 1 -- 172.21.15.52:0/2715387777 learned_addr learned my addr 172.21.15.52:0/2715387777 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:28.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a37fe640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38a415bf50 0x7f38a4150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f389bfff640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38a40aa550 0x7f38a4150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a2ffd640 1 -- 172.21.15.52:0/2715387777 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38a415bf50 msgr2=0x7f38a4150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a2ffd640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38a415bf50 0x7f38a4150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a2ffd640 1 -- 172.21.15.52:0/2715387777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38a40aa550 msgr2=0x7f38a4150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a2ffd640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38a40aa550 0x7f38a4150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a2ffd640 1 -- 172.21.15.52:0/2715387777 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a4151380 con 0x7f38a40aa0c0 2024-05-04T12:19:28.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f389bfff640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38a40aa550 0x7f38a4150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:19:28.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a2ffd640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38a40aa0c0 0x7f38a4152d10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f38940ad040 tx=0x7f389401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:28.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f389b7fe640 1 -- 172.21.15.52:0/2715387777 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38940bf610 con 0x7f38a40aa0c0 2024-05-04T12:19:28.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f389b7fe640 1 -- 172.21.15.52:0/2715387777 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38940bf7d0 con 0x7f38a40aa0c0 2024-05-04T12:19:28.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a9316640 1 -- 172.21.15.52:0/2715387777 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a4151660 con 0x7f38a40aa0c0 2024-05-04T12:19:28.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f389b7fe640 1 -- 172.21.15.52:0/2715387777 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38940bfbc0 con 0x7f38a40aa0c0 2024-05-04T12:19:28.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.425+0000 7f38a9316640 1 -- 172.21.15.52:0/2715387777 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a4133aa0 con 0x7f38a40aa0c0 2024-05-04T12:19:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f389b7fe640 1 -- 172.21.15.52:0/2715387777 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f38940bd020 con 0x7f38a40aa0c0 2024-05-04T12:19:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f389b7fe640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f38840678c0 0x7f3884069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f389b7fe640 1 -- 172.21.15.52:0/2715387777 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f38941201f0 con 0x7f38a40aa0c0 2024-05-04T12:19:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f389bfff640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f38840678c0 0x7f3884069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f389bfff640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f38840678c0 0x7f3884069d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f38a41f8900 tx=0x7f388c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 Processor -- start 2024-05-04T12:19:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 -- start start 2024-05-04T12:19:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd126912d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12691810 0x55fd12647680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12647bc0 0x55fd12647fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fd124955e0 con 0x55fd12690ef0 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fd1241cc50 con 0x55fd12647bc0 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x55fd124915a0 con 0x55fd12691810 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38a2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd126912d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38a2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd126912d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59630/0 (socket says 172.21.15.52:59630) 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38a2ffd640 1 -- 172.21.15.52:0/2444032164 learned_addr learned my addr 172.21.15.52:0/2444032164 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38a2ffd640 1 -- 172.21.15.52:0/2444032164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12691810 msgr2=0x55fd12647680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38a2ffd640 1 --2- 172.21.15.52:0/2444032164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12691810 0x55fd12647680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38a2ffd640 1 -- 172.21.15.52:0/2444032164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12647bc0 msgr2=0x55fd12647fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38a2ffd640 1 --2- 172.21.15.52:0/2444032164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12647bc0 0x55fd12647fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38a2ffd640 1 -- 172.21.15.52:0/2444032164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd1241ad10 con 0x55fd12690ef0 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38a2ffd640 1 --2- 172.21.15.52:0/2444032164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd126912d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38940f8ce0 tx=0x7f38940191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849e8dbf78604b6f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f3898ff9640 1 -- 172.21.15.52:0/2444032164 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38940f0630 con 0x55fd12690ef0 2024-05-04T12:19:28.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f3898ff9640 1 -- 172.21.15.52:0/2444032164 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38940b0050 con 0x55fd12690ef0 2024-05-04T12:19:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f3898ff9640 1 -- 172.21.15.52:0/2444032164 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38940f49e0 con 0x55fd12690ef0 2024-05-04T12:19:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 -- 172.21.15.52:0/2444032164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 msgr2=0x55fd126912d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 --2- 172.21.15.52:0/2444032164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd126912d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38940f8ce0 tx=0x7f38940191f0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 -- 172.21.15.52:0/2444032164 shutdown_connections 2024-05-04T12:19:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 --2- 172.21.15.52:0/2444032164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12647bc0 0x55fd12647fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 --2- 172.21.15.52:0/2444032164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12691810 0x55fd12647680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 --2- 172.21.15.52:0/2444032164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd126912d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.429+0000 7f38ac48f000 1 -- 172.21.15.52:0/2444032164 >> 172.21.15.52:0/2444032164 conn(0x55fd12643b70 msgr2=0x55fd12645f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 1 -- 172.21.15.52:0/2444032164 shutdown_connections 2024-05-04T12:19:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 1 -- 172.21.15.52:0/2444032164 wait complete. 2024-05-04T12:19:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 1 -- start start 2024-05-04T12:19:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 1 Processor -- start 2024-05-04T12:19:28.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12647bc0 0x55fd12675f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12690ef0 0x55fd12676460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12691810 0x55fd126769a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fd124955e0 con 0x55fd12691810 2024-05-04T12:19:28.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fd1244f520 con 0x55fd12690ef0 2024-05-04T12:19:28.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x55fd12462340 con 0x55fd12647bc0 2024-05-04T12:19:28.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 10 client.0 _ll_register_callbacks cb 0x55fd126325c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T12:19:28.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38a37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12691810 0x55fd126769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38a37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12691810 0x55fd126769a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59638/0 (socket says 172.21.15.52:59638) 2024-05-04T12:19:28.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38a37fe640 1 -- 172.21.15.52:0/2481996707 learned_addr learned my addr 172.21.15.52:0/2481996707 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:28.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38a2ffd640 1 --2- 172.21.15.52:0/2481996707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12647bc0 0x55fd12675f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f389bfff640 1 --2- 172.21.15.52:0/2481996707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12690ef0 0x55fd12676460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38a37fe640 1 -- 172.21.15.52:0/2481996707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12647bc0 msgr2=0x55fd12675f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:28.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38a37fe640 1 --2- 172.21.15.52:0/2481996707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12647bc0 0x55fd12675f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38a37fe640 1 -- 172.21.15.52:0/2481996707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12690ef0 msgr2=0x55fd12676460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:28.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38a37fe640 1 --2- 172.21.15.52:0/2481996707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12690ef0 0x55fd12676460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38a37fe640 1 -- 172.21.15.52:0/2481996707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd1242d7b0 con 0x55fd12691810 2024-05-04T12:19:28.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38a37fe640 1 --2- 172.21.15.52:0/2481996707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12691810 0x55fd126769a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f389c00e810 tx=0x7f389c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49df0c307de27d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:28.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f3881ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-04T12:19:28.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f3881ffb640 1 -- 172.21.15.52:0/2481996707 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c00c8d0 con 0x55fd12691810 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f3881ffb640 1 -- 172.21.15.52:0/2481996707 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f389c00ca70 con 0x55fd12691810 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 1 -- 172.21.15.52:0/2481996707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38940b0050 con 0x55fd12691810 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f3881ffb640 1 -- 172.21.15.52:0/2481996707 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f389c020050 con 0x55fd12691810 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 10 client.5382 fetch_fsmap learned FSMap version 16 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 1 -- 172.21.15.52:0/2481996707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55fd1241cea0 con 0x55fd12691810 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f3881ffb640 1 -- 172.21.15.52:0/2481996707 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38940b0050 con 0x55fd12691810 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f3881ffb640 1 -- 172.21.15.52:0/2481996707 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f389c023020 con 0x55fd12691810 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 10 client.5382 fetch_fsmap finished waiting for FSMap version 16 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 10 client.5382 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 20 client.5382 populate_metadata read hostname 'smithi052' 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 1 --2- 172.21.15.52:0/2481996707 >> [v2:172.21.15.52:6836/1620779861,v1:172.21.15.52:6837/1620779861] conn(0x55fd126a0f90 0x55fd126a3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 4 client.5382 mds_command: new command op to 5199 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38ac48f000 1 -- 172.21.15.52:0/2481996707 --> [v2:172.21.15.52:6836/1620779861,v1:172.21.15.52:6837/1620779861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd12430ab0 con 0x55fd126a0f90 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38a2ffd640 1 --2- 172.21.15.52:0/2481996707 >> [v2:172.21.15.52:6836/1620779861,v1:172.21.15.52:6837/1620779861] conn(0x55fd126a0f90 0x55fd126a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f3881ffb640 10 client.5382 ms_handle_connect on v2:172.21.15.52:6836/1620779861 2024-05-04T12:19:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.433+0000 7f38a2ffd640 1 --2- 172.21.15.52:0/2481996707 >> [v2:172.21.15.52:6836/1620779861,v1:172.21.15.52:6837/1620779861] conn(0x55fd126a0f90 0x55fd126a3380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3f6661fbb80c8b server_cookie=fdaf95edae7df727 in_seq=0 out_seq=0 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f3881ffb640 1 -- 172.21.15.52:0/2481996707 <== mds.0 v2:172.21.15.52:6836/1620779861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd12430ab0 con 0x55fd126a0f90 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f3881ffb640 10 client.5382 handle_command_reply: tid=0 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f38ac48f000 1 client.5382 shutdown 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f38ac48f000 1 -- 172.21.15.52:0/2481996707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12691810 msgr2=0x55fd126769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f38ac48f000 1 --2- 172.21.15.52:0/2481996707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12691810 0x55fd126769a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f389c00e810 tx=0x7f389c00ecd0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f38ac48f000 1 -- 172.21.15.52:0/2481996707 shutdown_connections 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f38ac48f000 1 --2- 172.21.15.52:0/2481996707 >> [v2:172.21.15.52:6836/1620779861,v1:172.21.15.52:6837/1620779861] conn(0x55fd126a0f90 0x55fd126a3380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f38ac48f000 1 --2- 172.21.15.52:0/2481996707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12691810 0x55fd126769a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f3881ffb640 0 client.5382 ms_handle_reset on v2:172.21.15.52:6836/1620779861 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f38ac48f000 1 --2- 172.21.15.52:0/2481996707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12690ef0 0x55fd12676460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f38ac48f000 1 --2- 172.21.15.52:0/2481996707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12647bc0 0x55fd12675f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f38ac48f000 1 -- 172.21.15.52:0/2481996707 >> 172.21.15.52:0/2481996707 conn(0x55fd12643b70 msgr2=0x55fd126467f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f38ac48f000 1 -- 172.21.15.52:0/2481996707 shutdown_connections 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f38ac48f000 1 -- 172.21.15.52:0/2481996707 wait complete. 2024-05-04T12:19:28.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.437+0000 7f38ac48f000 20 client.5382 trim_cache size 0 max 16384 2024-05-04T12:19:28.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.453+0000 7f38ac48f000 1 Processor -- start 2024-05-04T12:19:28.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.453+0000 7f38ac48f000 1 -- start start 2024-05-04T12:19:28.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.453+0000 7f38ac48f000 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12647bc0 0x55fd126a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.453+0000 7f38ac48f000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd126a5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.453+0000 7f38ac48f000 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12691810 0x55fd126a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.453+0000 7f38ac48f000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f38a4200330 con 0x55fd12690ef0 2024-05-04T12:19:28.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.453+0000 7f38ac48f000 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f38a41363b0 con 0x55fd12691810 2024-05-04T12:19:28.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.453+0000 7f38ac48f000 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f38a41ffd10 con 0x55fd12647bc0 2024-05-04T12:19:28.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd126a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd126a5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59642/0 (socket says 172.21.15.52:59642) 2024-05-04T12:19:28.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 -- 172.21.15.52:0/4131099821 learned_addr learned my addr 172.21.15.52:0/4131099821 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:28.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 -- 172.21.15.52:0/4131099821 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12647bc0 msgr2=0x55fd126a66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:28.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38a2ffd640 1 --2- 172.21.15.52:0/4131099821 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12647bc0 0x55fd126a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38a37fe640 1 --2- 172.21.15.52:0/4131099821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12691810 0x55fd126a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 --2- 172.21.15.52:0/4131099821 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12647bc0 0x55fd126a66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 -- 172.21.15.52:0/4131099821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12691810 msgr2=0x55fd126a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:28.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 --2- 172.21.15.52:0/4131099821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12691810 0x55fd126a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 -- 172.21.15.52:0/4131099821 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd1241de80 con 0x55fd12690ef0 2024-05-04T12:19:28.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 --2- 172.21.15.52:0/4131099821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd126a5780 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f388c006f10 tx=0x7f388c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc485625f13be0d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:28.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f3883fff640 1 -- 172.21.15.52:0/4131099821 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c027070 con 0x55fd12690ef0 2024-05-04T12:19:28.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f3883fff640 1 -- 172.21.15.52:0/4131099821 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388c003820 con 0x55fd12690ef0 2024-05-04T12:19:28.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f3883fff640 1 -- 172.21.15.52:0/4131099821 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c00f040 con 0x55fd12690ef0 2024-05-04T12:19:28.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 -- 172.21.15.52:0/4131099821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 msgr2=0x55fd126a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 --2- 172.21.15.52:0/4131099821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd126a5780 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f388c006f10 tx=0x7f388c004740 comp rx=0 tx=0).stop 2024-05-04T12:19:28.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 -- 172.21.15.52:0/4131099821 shutdown_connections 2024-05-04T12:19:28.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 --2- 172.21.15.52:0/4131099821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12691810 0x55fd126a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 --2- 172.21.15.52:0/4131099821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd126a5780 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 --2- 172.21.15.52:0/4131099821 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12647bc0 0x55fd126a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 -- 172.21.15.52:0/4131099821 >> 172.21.15.52:0/4131099821 conn(0x55fd12643b70 msgr2=0x55fd1268f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:28.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 -- 172.21.15.52:0/4131099821 shutdown_connections 2024-05-04T12:19:28.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 -- 172.21.15.52:0/4131099821 wait complete. 2024-05-04T12:19:28.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 -- start start 2024-05-04T12:19:28.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 Processor -- start 2024-05-04T12:19:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12647bc0 0x55fd12705dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd12708310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12691810 0x55fd1270a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fd1241d640 con 0x55fd12690ef0 2024-05-04T12:19:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fd1245e810 con 0x55fd12647bc0 2024-05-04T12:19:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x55fd1245e3c0 con 0x55fd12691810 2024-05-04T12:19:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 10 client.0 _ll_register_callbacks cb 0x7f389c001a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T12:19:28.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd12708310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd12708310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59654/0 (socket says 172.21.15.52:59654) 2024-05-04T12:19:28.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 -- 172.21.15.52:0/524260479 learned_addr learned my addr 172.21.15.52:0/524260479 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:28.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 -- 172.21.15.52:0/524260479 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12691810 msgr2=0x55fd1270a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:28.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 --2- 172.21.15.52:0/524260479 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12691810 0x55fd1270a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 -- 172.21.15.52:0/524260479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12647bc0 msgr2=0x55fd12705dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:28.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 --2- 172.21.15.52:0/524260479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12647bc0 0x55fd12705dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 -- 172.21.15.52:0/524260479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd12462760 con 0x55fd12690ef0 2024-05-04T12:19:28.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f389bfff640 1 --2- 172.21.15.52:0/524260479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd12708310 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x55fd121b9ec0 tx=0x7f388c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0129d71cbf548b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:28.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f3881ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-04T12:19:28.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f3881ffb640 1 -- 172.21.15.52:0/524260479 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c030020 con 0x55fd12690ef0 2024-05-04T12:19:28.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f3881ffb640 1 -- 172.21.15.52:0/524260479 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f388c013430 con 0x55fd12690ef0 2024-05-04T12:19:28.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 -- 172.21.15.52:0/524260479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f388c003820 con 0x55fd12690ef0 2024-05-04T12:19:28.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f3881ffb640 1 -- 172.21.15.52:0/524260479 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f388c03a050 con 0x55fd12690ef0 2024-05-04T12:19:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 10 client.5388 fetch_fsmap learned FSMap version 16 2024-05-04T12:19:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.457+0000 7f38ac48f000 1 -- 172.21.15.52:0/524260479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55fd124ee9e0 con 0x55fd12690ef0 2024-05-04T12:19:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f3881ffb640 1 -- 172.21.15.52:0/524260479 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388c003820 con 0x55fd12690ef0 2024-05-04T12:19:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f3881ffb640 1 -- 172.21.15.52:0/524260479 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f388c03d020 con 0x55fd12690ef0 2024-05-04T12:19:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 10 client.5388 fetch_fsmap finished waiting for FSMap version 16 2024-05-04T12:19:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 10 client.5388 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-04T12:19:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 20 client.5388 populate_metadata read hostname 'smithi052' 2024-05-04T12:19:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 1 --2- 172.21.15.52:0/524260479 >> [v2:172.21.15.52:6836/1620779861,v1:172.21.15.52:6837/1620779861] conn(0x55fd1270ada0 0x55fd1270d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 4 client.5388 mds_command: new command op to 5199 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T12:19:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 1 -- 172.21.15.52:0/524260479 --> [v2:172.21.15.52:6836/1620779861,v1:172.21.15.52:6837/1620779861] -- command(tid 0: {"prefix": "status"}) -- 0x55fd124a2b20 con 0x55fd1270ada0 2024-05-04T12:19:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38a2ffd640 1 --2- 172.21.15.52:0/524260479 >> [v2:172.21.15.52:6836/1620779861,v1:172.21.15.52:6837/1620779861] conn(0x55fd1270ada0 0x55fd1270d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f3881ffb640 10 client.5388 ms_handle_connect on v2:172.21.15.52:6836/1620779861 2024-05-04T12:19:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38a2ffd640 1 --2- 172.21.15.52:0/524260479 >> [v2:172.21.15.52:6836/1620779861,v1:172.21.15.52:6837/1620779861] conn(0x55fd1270ada0 0x55fd1270d190 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4329771da2190ec server_cookie=4355ed77ef5c16b5 in_seq=0 out_seq=0 2024-05-04T12:19:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f3881ffb640 1 -- 172.21.15.52:0/524260479 <== mds.0 v2:172.21.15.52:6836/1620779861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55fd124a2b20 con 0x55fd1270ada0 2024-05-04T12:19:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f3881ffb640 10 client.5388 handle_command_reply: tid=0 2024-05-04T12:19:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 1 client.5388 shutdown 2024-05-04T12:19:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 1 -- 172.21.15.52:0/524260479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 msgr2=0x55fd12708310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 1 --2- 172.21.15.52:0/524260479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd12708310 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x55fd121b9ec0 tx=0x7f388c014000 comp rx=0 tx=0).stop 2024-05-04T12:19:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 1 -- 172.21.15.52:0/524260479 shutdown_connections 2024-05-04T12:19:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 1 --2- 172.21.15.52:0/524260479 >> [v2:172.21.15.52:6836/1620779861,v1:172.21.15.52:6837/1620779861] conn(0x55fd1270ada0 0x55fd1270d190 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 1 --2- 172.21.15.52:0/524260479 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55fd12691810 0x55fd1270a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 1 --2- 172.21.15.52:0/524260479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd12690ef0 0x55fd12708310 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 1 --2- 172.21.15.52:0/524260479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd12647bc0 0x55fd12705dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f3881ffb640 0 client.5388 ms_handle_reset on v2:172.21.15.52:6836/1620779861 2024-05-04T12:19:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 1 -- 172.21.15.52:0/524260479 >> 172.21.15.52:0/524260479 conn(0x55fd12643b70 msgr2=0x55fd126a6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 1 -- 172.21.15.52:0/524260479 shutdown_connections 2024-05-04T12:19:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 1 -- 172.21.15.52:0/524260479 wait complete. 2024-05-04T12:19:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38ac48f000 20 client.5388 trim_cache size 0 max 16384 2024-05-04T12:19:28.471 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-04T12:19:28.471 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "708d9177-9f3e-4c4b-9d29-7b98768b2a25", 2024-05-04T12:19:28.471 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-05-04T12:19:28.471 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 5199, 2024-05-04T12:19:28.471 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-05-04T12:19:28.471 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-05-04T12:19:28.471 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-05-04T12:19:28.471 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 5.7902633699999999, 2024-05-04T12:19:28.471 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 16, 2024-05-04T12:19:28.471 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 34, 2024-05-04T12:19:28.472 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 34, 2024-05-04T12:19:28.472 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 12.717856134 2024-05-04T12:19:28.472 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-04T12:19:28.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38a9316640 1 -- 172.21.15.52:0/2715387777 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f38840678c0 msgr2=0x7f3884069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38a9316640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f38840678c0 0x7f3884069d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f38a41f8900 tx=0x7f388c008040 comp rx=0 tx=0).stop 2024-05-04T12:19:28.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38a9316640 1 -- 172.21.15.52:0/2715387777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38a40aa0c0 msgr2=0x7f38a4152d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38a9316640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38a40aa0c0 0x7f38a4152d10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f38940ad040 tx=0x7f389401ac90 comp rx=0 tx=0).stop 2024-05-04T12:19:28.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38a9316640 1 -- 172.21.15.52:0/2715387777 shutdown_connections 2024-05-04T12:19:28.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38a9316640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f38840678c0 0x7f3884069d80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38a9316640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38a415bf50 0x7f38a4150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38a9316640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38a40aa550 0x7f38a4150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38a9316640 1 --2- 172.21.15.52:0/2715387777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38a40aa0c0 0x7f38a4152d10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38a9316640 1 -- 172.21.15.52:0/2715387777 >> 172.21.15.52:0/2715387777 conn(0x7f38a40b14a0 msgr2=0x7f38a40a9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:28.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38a9316640 1 -- 172.21.15.52:0/2715387777 shutdown_connections 2024-05-04T12:19:28.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.461+0000 7f38a9316640 1 -- 172.21.15.52:0/2715387777 wait complete. 2024-05-04T12:19:28.481 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-04T12:19:28.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.589+0000 7f23d4d6e640 1 Processor -- start 2024-05-04T12:19:28.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23d4d6e640 1 -- start start 2024-05-04T12:19:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23d4d6e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23d015bf50 0x7f23d015c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23d4d6e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f23d00aa0c0 0x7f23d00aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23d4d6e640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23d006d1b0 con 0x7f23d00ccb50 2024-05-04T12:19:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23d4d6e640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f23d006d490 con 0x7f23d00aa0c0 2024-05-04T12:19:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23d4d6e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f23d006d770 con 0x7f23d015bf50 2024-05-04T12:19:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23cdd9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f23d00aa0c0 0x7f23d00aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23cdd9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f23d00aa0c0 0x7f23d00aa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59668/0 (socket says 172.21.15.52:59668) 2024-05-04T12:19:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23ced9d640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f23d00ccb50 0x7f23d006e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:41126/0 (socket says 172.21.15.52:41126) 2024-05-04T12:19:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23cdd9b640 1 -- 172.21.15.52:0/2005996247 learned_addr learned my addr 172.21.15.52:0/2005996247 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23ce59c640 1 --2- 172.21.15.52:0/2005996247 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23d015bf50 0x7f23d015c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23cdd9b640 1 -- 172.21.15.52:0/2005996247 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23d015bf50 msgr2=0x7f23d015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:19:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23cdd9b640 1 --2- 172.21.15.52:0/2005996247 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23d015bf50 0x7f23d015c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23cdd9b640 1 -- 172.21.15.52:0/2005996247 >> v1:172.21.15.52:6789/0 conn(0x7f23d00ccb50 legacy=0x7f23d006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23cdd9b640 1 -- 172.21.15.52:0/2005996247 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d006da50 con 0x7f23d00aa0c0 2024-05-04T12:19:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23cd59a640 1 -- 172.21.15.52:0/2005996247 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3536493722 0 0) 0x7f23d006d1b0 con 0x7f23d00ccb50 2024-05-04T12:19:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23cdd9b640 1 --2- 172.21.15.52:0/2005996247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23d00aa0c0 0x7f23d00aa520 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009870 tx=0x7f23c401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729025d461086aaf server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23cd59a640 1 -- 172.21.15.52:0/2005996247 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c40a9610 con 0x7f23d00aa0c0 2024-05-04T12:19:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23cd59a640 1 -- 172.21.15.52:0/2005996247 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c40a97b0 con 0x7f23d00aa0c0 2024-05-04T12:19:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.593+0000 7f23cd59a640 1 -- 172.21.15.52:0/2005996247 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c40a9b80 con 0x7f23d00aa0c0 2024-05-04T12:19:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2005996247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23d00aa0c0 msgr2=0x7f23d00aa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 --2- 172.21.15.52:0/2005996247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23d00aa0c0 0x7f23d00aa520 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009870 tx=0x7f23c401c530 comp rx=0 tx=0).stop 2024-05-04T12:19:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2005996247 shutdown_connections 2024-05-04T12:19:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 --2- 172.21.15.52:0/2005996247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23d00aa0c0 0x7f23d00aa520 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 --2- 172.21.15.52:0/2005996247 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23d015bf50 0x7f23d015c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2005996247 >> 172.21.15.52:0/2005996247 conn(0x7f23d00b14a0 msgr2=0x7f23d00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2005996247 shutdown_connections 2024-05-04T12:19:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2005996247 wait complete. 2024-05-04T12:19:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 Processor -- start 2024-05-04T12:19:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 -- start start 2024-05-04T12:19:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23d00aa0c0 0x7f23d00cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23d00ccb50 0x7f23d00b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23d015bf50 0x7f23d00b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f23d006cee0 con 0x7f23d00aa0c0 2024-05-04T12:19:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f23d006cd60 con 0x7f23d00ccb50 2024-05-04T12:19:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f23d0169030 con 0x7f23d015bf50 2024-05-04T12:19:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23ce59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23d00aa0c0 0x7f23d00cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23ce59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23d00aa0c0 0x7f23d00cfa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59670/0 (socket says 172.21.15.52:59670) 2024-05-04T12:19:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23ce59c640 1 -- 172.21.15.52:0/2582097023 learned_addr learned my addr 172.21.15.52:0/2582097023 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:19:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23ce59c640 1 -- 172.21.15.52:0/2582097023 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23d015bf50 msgr2=0x7f23d00b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23ce59c640 1 --2- 172.21.15.52:0/2582097023 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23d015bf50 0x7f23d00b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23ce59c640 1 -- 172.21.15.52:0/2582097023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23d00ccb50 msgr2=0x7f23d00b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23ce59c640 1 --2- 172.21.15.52:0/2582097023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23d00ccb50 0x7f23d00b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23ce59c640 1 -- 172.21.15.52:0/2582097023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d00ceff0 con 0x7f23d00aa0c0 2024-05-04T12:19:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23ce59c640 1 --2- 172.21.15.52:0/2582097023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23d00aa0c0 0x7f23d00cfa90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f23c8007e40 tx=0x7f23c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23beffd640 1 -- 172.21.15.52:0/2582097023 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8014620 con 0x7f23d00aa0c0 2024-05-04T12:19:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23beffd640 1 -- 172.21.15.52:0/2582097023 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c8019070 con 0x7f23d00aa0c0 2024-05-04T12:19:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2582097023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d00710a0 con 0x7f23d00aa0c0 2024-05-04T12:19:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23beffd640 1 -- 172.21.15.52:0/2582097023 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8014a00 con 0x7f23d00aa0c0 2024-05-04T12:19:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.597+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2582097023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d0071660 con 0x7f23d00aa0c0 2024-05-04T12:19:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.601+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2582097023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d0062e50 con 0x7f23d00aa0c0 2024-05-04T12:19:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.601+0000 7f23beffd640 1 -- 172.21.15.52:0/2582097023 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f23c8014ba0 con 0x7f23d00aa0c0 2024-05-04T12:19:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.601+0000 7f23beffd640 1 --2- 172.21.15.52:0/2582097023 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f23b00678c0 0x7f23b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:19:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.601+0000 7f23beffd640 1 -- 172.21.15.52:0/2582097023 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f23c8086580 con 0x7f23d00aa0c0 2024-05-04T12:19:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.601+0000 7f23cdd9b640 1 --2- 172.21.15.52:0/2582097023 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f23b00678c0 0x7f23b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:19:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.601+0000 7f23cdd9b640 1 --2- 172.21.15.52:0/2582097023 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f23b00678c0 0x7f23b0069d80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f23c40ae000 tx=0x7f23c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:19:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.605+0000 7f23beffd640 1 -- 172.21.15.52:0/2582097023 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f23c80570b0 con 0x7f23d00aa0c0 2024-05-04T12:19:28.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.805+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2582097023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f23d00ce3f0 con 0x7f23d00aa0c0 2024-05-04T12:19:28.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.805+0000 7f23beffd640 1 -- 172.21.15.52:0/2582097023 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f23c80569b0 con 0x7f23d00aa0c0 2024-05-04T12:19:28.811 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-04T12:19:28.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.809+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2582097023 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f23b00678c0 msgr2=0x7f23b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.809+0000 7f23d4d6e640 1 --2- 172.21.15.52:0/2582097023 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f23b00678c0 0x7f23b0069d80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f23c40ae000 tx=0x7f23c4008040 comp rx=0 tx=0).stop 2024-05-04T12:19:28.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.809+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2582097023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23d00aa0c0 msgr2=0x7f23d00cfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:19:28.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.809+0000 7f23d4d6e640 1 --2- 172.21.15.52:0/2582097023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23d00aa0c0 0x7f23d00cfa90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f23c8007e40 tx=0x7f23c800e4b0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.809+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2582097023 shutdown_connections 2024-05-04T12:19:28.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.809+0000 7f23d4d6e640 1 --2- 172.21.15.52:0/2582097023 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f23b00678c0 0x7f23b0069d80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.809+0000 7f23d4d6e640 1 --2- 172.21.15.52:0/2582097023 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23d015bf50 0x7f23d00b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.809+0000 7f23d4d6e640 1 --2- 172.21.15.52:0/2582097023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23d00ccb50 0x7f23d00b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.809+0000 7f23d4d6e640 1 --2- 172.21.15.52:0/2582097023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23d00aa0c0 0x7f23d00cfa90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:19:28.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.809+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2582097023 >> 172.21.15.52:0/2582097023 conn(0x7f23d00b14a0 msgr2=0x7f23d0159490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:19:28.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.809+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2582097023 shutdown_connections 2024-05-04T12:19:28.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:28.809+0000 7f23d4d6e640 1 -- 172.21.15.52:0/2582097023 wait complete. 2024-05-04T12:19:28.828 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-04T12:19:28.828 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-04T12:19:28.828 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-05-04T12:19:28.828 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-04T12:19:28.828 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-04T12:19:28.828 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-04T12:19:28.828 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-04T12:19:28.828 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-04T12:19:28.828 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-04T12:19:28.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:28.829 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-05-04T12:19:28.838 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-04T12:19:28.838 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-04T12:19:28.838 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-05-04T12:19:28.838 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:19:28.838 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-05-04T12:19:28.839 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:19:28.839 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:19:28.839 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:19:28.839 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-05-04T12:19:28.839 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:19:28.840 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:19:28.840 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-05-04T12:19:28.840 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-04T12:19:28.840 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-05-04T12:19:28.840 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-05-04T12:19:28.840 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-04T12:19:28.840 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42447sec preferred_lft 42447sec 2024-05-04T12:19:28.840 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-05-04T12:19:28.840 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:19:28.840 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-04T12:19:28.840 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-05-04T12:19:28.841 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-05-04T12:19:28.841 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-04T12:19:28.841 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:41:0c:ed:b4 brd ff:ff:ff:ff:ff:ff 2024-05-04T12:19:28.841 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-04T12:19:28.841 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:19:28.841 INFO:teuthology.orchestra.run.smithi052.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-04T12:19:28.841 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-04T12:19:28.841 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-04T12:19:28.841 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:19:28.841 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-04T12:19:28.841 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:19:28.841 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-04T12:19:28.841 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:19:28.842 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-04T12:19:28.842 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:19:28.842 INFO:teuthology.orchestra.run.smithi052.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-04T12:19:28.842 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T12:19:28.842 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-04T12:19:28.842 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-04T12:19:28.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:28.843 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-04T12:19:28.889 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-05-04T12:19:28.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:28.891 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-05-04T12:19:28.941 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-05-04T12:19:28.941 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-04T12:19:28.941 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-05-04T12:19:28.941 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-04T12:19:28.941 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-05-04T12:19:28.941 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-04T12:19:28.941 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T12:19:28.942 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-04T12:19:28.942 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-04T12:19:28.942 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-04T12:19:28.942 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-04T12:19:28.942 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-04T12:19:29.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T12:19:29.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:19:29.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:29.067 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-05-04T12:19:29.072 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-04T12:19:29.073 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-04T12:19:29.073 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-04T12:19:29.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:29.073 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-05-04T12:19:29.127 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-04T12:19:29.127 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-04T12:19:29.127 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-04T12:19:29.175 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-04T12:19:29.176 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-04T12:19:29.176 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-04T12:19:29.224 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-04T12:19:29.281 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:19:29.330 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/run/lock 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-04T12:19:29.331 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/fs/nfsd 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/run/rpc_pipefs 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:19:29.332 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:19:29.333 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T12:19:29.333 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T12:19:29.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:29.333 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-04T12:19:29.380 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-04T12:19:29.380 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-04T12:19:29.425 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-04T12:19:29.439 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:19:29.488 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-04T12:19:29.488 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-04T12:19:29.488 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-04T12:19:29.488 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-04T12:19:29.488 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-04T12:19:29.488 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-04T12:19:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-04T12:19:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-04T12:19:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:/run/lock 2024-05-04T12:19:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-04T12:19:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-04T12:19:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-04T12:19:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:19:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-04T12:19:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-04T12:19:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-04T12:19:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-04T12:19:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/fs/nfsd 2024-05-04T12:19:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-04T12:19:29.489 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-04T12:19:29.490 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:19:29.490 INFO:teuthology.orchestra.run.smithi052.stdout:/run/rpc_pipefs 2024-05-04T12:19:29.490 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-04T12:19:29.490 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-04T12:19:29.490 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-04T12:19:29.490 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-04T12:19:29.491 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-04T12:19:29.491 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-04T12:19:29.491 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-04T12:19:29.491 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:19:29.491 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:19:29.491 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T12:19:29.491 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T12:19:29.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:29.491 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-04T12:19:29.509 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:2024-05-04T12:19:29.505+0000 7f73fbe04580 -1 init, newargv = 0x560366b56af0 newargc=15 2024-05-04T12:19:29.509 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[37511]: starting ceph client 2024-05-04T12:19:29.514 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[37511]: starting fuse 2024-05-04T12:19:30.615 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-04T12:19:30.630 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T12:19:30.683 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-04T12:19:30.683 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-04T12:19:30.683 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-04T12:19:30.683 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-04T12:19:30.683 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-04T12:19:30.683 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-04T12:19:30.683 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-04T12:19:30.683 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-04T12:19:30.683 INFO:teuthology.orchestra.run.smithi052.stdout:/run/lock 2024-05-04T12:19:30.683 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-04T12:19:30.683 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/fs/nfsd 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/run/rpc_pipefs 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-04T12:19:30.684 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-04T12:19:30.685 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-04T12:19:30.685 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-04T12:19:30.685 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-04T12:19:30.685 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:19:30.685 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T12:19:30.685 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T12:19:30.685 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T12:19:30.685 INFO:teuthology.orchestra.run.smithi052.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-04T12:19:30.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:30.686 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-04T12:19:30.732 INFO:teuthology.orchestra.run.smithi052.stdout:45 2024-05-04T12:19:30.732 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-04T12:19:30.732 DEBUG:teuthology.orchestra.run.smithi052:> sudo stdin-killer -- python3 -c ' 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> import glob 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> import re 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> import os 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> import subprocess 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> def _find_admin_socket(client_name): 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> files = glob.glob(asok_path) 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> # Given a non-glob path, it better be there 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> if "*" not in asok_path: 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> assert(len(files) == 1) 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> return files[0] 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> for f in files: 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-04T12:19:30.733 DEBUG:teuthology.orchestra.run.smithi052:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-04T12:19:30.734 DEBUG:teuthology.orchestra.run.smithi052:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-04T12:19:30.734 DEBUG:teuthology.orchestra.run.smithi052:> contents = proc_f.read() 2024-05-04T12:19:30.734 DEBUG:teuthology.orchestra.run.smithi052:> if mountpoint in contents: 2024-05-04T12:19:30.734 DEBUG:teuthology.orchestra.run.smithi052:> return f 2024-05-04T12:19:30.734 DEBUG:teuthology.orchestra.run.smithi052:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-04T12:19:30.734 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-04T12:19:30.734 DEBUG:teuthology.orchestra.run.smithi052:> print(_find_admin_socket("client.0")) 2024-05-04T12:19:30.734 DEBUG:teuthology.orchestra.run.smithi052:> ' 2024-05-04T12:19:30.862 INFO:teuthology.orchestra.run.smithi052.stdout:/var/run/ceph/ceph-client.0.37511.asok 2024-05-04T12:19:30.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:19:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T12:19:30.875 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37511.asok 2024-05-04T12:19:30.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:30.875 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37511.asok status 2024-05-04T12:19:31.039 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-04T12:19:31.040 INFO:teuthology.orchestra.run.smithi052.stdout: "metadata": { 2024-05-04T12:19:31.040 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_sha1": "09dbd6bba52d19ba471c33bc15c008e6ad158ea6", 2024-05-04T12:19:31.040 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-2455-g09dbd6bb (09dbd6bba52d19ba471c33bc15c008e6ad158ea6) squid (dev)", 2024-05-04T12:19:31.040 INFO:teuthology.orchestra.run.smithi052.stdout: "entity_id": "0", 2024-05-04T12:19:31.040 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": "smithi052", 2024-05-04T12:19:31.040 INFO:teuthology.orchestra.run.smithi052.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-04T12:19:31.040 INFO:teuthology.orchestra.run.smithi052.stdout: "pid": "37511", 2024-05-04T12:19:31.040 INFO:teuthology.orchestra.run.smithi052.stdout: "root": "/" 2024-05-04T12:19:31.040 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:31.040 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_count": 0, 2024-05-04T12:19:31.040 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_pinned_count": 0, 2024-05-04T12:19:31.040 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 5406, 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: "inst": { 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: "name": { 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "client", 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: "num": 5406 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1457634161 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1457634161 2024-05-04T12:19:31.041 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-04T12:19:31.042 INFO:teuthology.orchestra.run.smithi052.stdout: "inst_str": "client.5406 192.168.0.1:0/1457634161", 2024-05-04T12:19:31.042 INFO:teuthology.orchestra.run.smithi052.stdout: "addr_str": "192.168.0.1:0/1457634161", 2024-05-04T12:19:31.042 INFO:teuthology.orchestra.run.smithi052.stdout: "inode_count": 1, 2024-05-04T12:19:31.042 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_epoch": 16, 2024-05-04T12:19:31.042 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch": 34, 2024-05-04T12:19:31.042 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch_barrier": 0, 2024-05-04T12:19:31.042 INFO:teuthology.orchestra.run.smithi052.stdout: "blocklisted": false, 2024-05-04T12:19:31.042 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs" 2024-05-04T12:19:31.042 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-04T12:19:31.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:31.055 DEBUG:teuthology.orchestra.run.smithi052:> stat --file-system '--printf=%T 2024-05-04T12:19:31.055 DEBUG:teuthology.orchestra.run.smithi052:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-04T12:19:31.062 INFO:teuthology.orchestra.run.smithi052.stdout:fuseblk 2024-05-04T12:19:31.062 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-04T12:19:31.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:31.062 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-04T12:19:31.116 DEBUG:teuthology.orchestra.run.smithi052:> mktemp -d --suffix=xfstests-dev 2024-05-04T12:19:31.164 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.BtfdD8Y1eWxfstests-dev 2024-05-04T12:19:31.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:31.164 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.BtfdD8Y1eWxfstests-dev) 2024-05-04T12:19:31.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:31.223 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.BtfdD8Y1eWxfstests-dev) 2024-05-04T12:19:31.281 INFO:teuthology.orchestra.run.smithi052.stderr:Cloning into '/tmp/tmp.BtfdD8Y1eWxfstests-dev'... 2024-05-04T12:19:32.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:32.076 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-05-04T12:19:32.132 DEBUG:teuthology.orchestra.run.smithi052:> mktemp -d --suffix=test 2024-05-04T12:19:32.180 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.FGpMFsgr0Qtest 2024-05-04T12:19:32.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:19:32.180 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-05-04T12:19:32.233 DEBUG:teuthology.orchestra.run.smithi052:> mktemp -d --suffix=scratch 2024-05-04T12:19:32.280 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.JrqTWk0bCcscratch 2024-05-04T12:19:32.280 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "ubuntu" and "22.04". 2024-05-04T12:19:32.280 DEBUG:teuthology.orchestra.run.smithi052:> sudo apt-get install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc git indent libacl1-dev libaio-dev libcap-dev libgdbm-dev libtool libtool-bin liburing-dev libuuid1 lvm2 make psmisc python3 quota sed uuid-dev uuid-runtime xfsprogs sqlite3 exfatprogs f2fs-tools ocfs2-tools udftools xfsdump xfslibs-dev 2024-05-04T12:19:32.396 INFO:teuthology.orchestra.run.smithi052.stdout:Reading package lists... 2024-05-04T12:19:32.604 INFO:teuthology.orchestra.run.smithi052.stdout:Building dependency tree... 2024-05-04T12:19:32.605 INFO:teuthology.orchestra.run.smithi052.stdout:Reading state information... 2024-05-04T12:19:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:acl is already the newest version (2.3.1-1). 2024-05-04T12:19:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:acl set to manually installed. 2024-05-04T12:19:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:attr is already the newest version (1:2.5.1-1build1). 2024-05-04T12:19:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:automake is already the newest version (1:1.16.5-1.3). 2024-05-04T12:19:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:bc is already the newest version (1.07.1-3build1). 2024-05-04T12:19:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:gcc is already the newest version (4:11.2.0-1ubuntu1). 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:gcc set to manually installed. 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:libacl1-dev is already the newest version (2.3.1-1). 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:libaio-dev is already the newest version (0.3.112-13build1). 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:libtool is already the newest version (2.4.6-15build2). 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:lvm2 is already the newest version (2.03.11-2.1ubuntu4). 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:make is already the newest version (4.3-4.1build1). 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:make set to manually installed. 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:psmisc is already the newest version (23.4-2build3). 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:quota is already the newest version (4.06-1build2). 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:sed is already the newest version (4.8-1ubuntu2). 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:xfsdump is already the newest version (3.1.9+0+nmu2). 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:xfsprogs is already the newest version (5.13.0-1ubuntu2). 2024-05-04T12:19:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:dbench is already the newest version (4.0-2build1). 2024-05-04T12:19:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:dump is already the newest version (0.4b47-3). 2024-05-04T12:19:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:e2fsprogs is already the newest version (1.46.5-2ubuntu1.1). 2024-05-04T12:19:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:git is already the newest version (1:2.34.1-1ubuntu1.10). 2024-05-04T12:19:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:indent is already the newest version (2.2.12-1ubuntu0.22.04.1). 2024-05-04T12:19:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:libuuid1 is already the newest version (2.37.2-4ubuntu3.4). 2024-05-04T12:19:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:python3 is already the newest version (3.10.6-1~22.04). 2024-05-04T12:19:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3.4). 2024-05-04T12:19:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:The following packages were automatically installed and are no longer required: 2024-05-04T12:19:32.815 INFO:teuthology.orchestra.run.smithi052.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libiscsi7 libpmemobj1 2024-05-04T12:19:32.815 INFO:teuthology.orchestra.run.smithi052.stdout: libpython2-dev libpython2-stdlib libpython2.7 libpython2.7-dev 2024-05-04T12:19:32.815 INFO:teuthology.orchestra.run.smithi052.stdout: libpython2.7-minimal libpython2.7-stdlib linux-image-5.4.0-135-generic 2024-05-04T12:19:32.816 INFO:teuthology.orchestra.run.smithi052.stdout: linux-modules-5.4.0-135-generic python2 python2-dev python2-minimal 2024-05-04T12:19:32.816 INFO:teuthology.orchestra.run.smithi052.stdout: python2.7 python2.7-dev python2.7-minimal 2024-05-04T12:19:32.816 INFO:teuthology.orchestra.run.smithi052.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-04T12:19:32.816 INFO:teuthology.orchestra.run.smithi052.stdout:The following additional packages will be installed: 2024-05-04T12:19:32.816 INFO:teuthology.orchestra.run.smithi052.stdout: libcap2 libcmap4 libcorosync-common4 libdlm3 libpmemblk1 libqb100 2024-05-04T12:19:32.817 INFO:teuthology.orchestra.run.smithi052.stdout: libsqlite3-0 2024-05-04T12:19:32.819 INFO:teuthology.orchestra.run.smithi052.stdout:Suggested packages: 2024-05-04T12:19:32.819 INFO:teuthology.orchestra.run.smithi052.stdout: gnuplot gfio python-scipy gawk-doc sqlite3-doc dvd+rw-tools udfclient 2024-05-04T12:19:32.852 INFO:teuthology.orchestra.run.smithi052.stdout:The following NEW packages will be installed: 2024-05-04T12:19:32.852 INFO:teuthology.orchestra.run.smithi052.stdout: exfatprogs f2fs-tools fio libcap-dev libcmap4 libcorosync-common4 libdlm3 2024-05-04T12:19:32.853 INFO:teuthology.orchestra.run.smithi052.stdout: libgdbm-dev libpmemblk1 libqb100 liburing-dev ocfs2-tools sqlite3 udftools 2024-05-04T12:19:32.854 INFO:teuthology.orchestra.run.smithi052.stdout:The following packages will be upgraded: 2024-05-04T12:19:32.855 INFO:teuthology.orchestra.run.smithi052.stdout: gawk libcap2 libsqlite3-0 uuid-runtime 2024-05-04T12:19:33.039 INFO:teuthology.orchestra.run.smithi052.stdout:4 upgraded, 14 newly installed, 0 to remove and 295 not upgraded. 2024-05-04T12:19:33.039 INFO:teuthology.orchestra.run.smithi052.stdout:Need to get 7,314 kB of archives. 2024-05-04T12:19:33.039 INFO:teuthology.orchestra.run.smithi052.stdout:After this operation, 14.2 MB of additional disk space will be used. 2024-05-04T12:19:33.039 INFO:teuthology.orchestra.run.smithi052.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-runtime amd64 2.37.2-4ubuntu3.4 [32.0 kB] 2024-05-04T12:19:33.247 INFO:teuthology.orchestra.run.smithi052.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 gawk amd64 1:5.1.0-1ubuntu0.1 [447 kB] 2024-05-04T12:19:33.586 INFO:teuthology.orchestra.run.smithi052.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 libcorosync-common4 amd64 3.1.6-1ubuntu1 [7,322 B] 2024-05-04T12:19:33.587 INFO:teuthology.orchestra.run.smithi052.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqb100 amd64 2.0.4-1ubuntu0.2 [74.2 kB] 2024-05-04T12:19:33.605 INFO:teuthology.orchestra.run.smithi052.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 libcmap4 amd64 3.1.6-1ubuntu1 [10.4 kB] 2024-05-04T12:19:33.607 INFO:teuthology.orchestra.run.smithi052.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 libdlm3 amd64 4.1.1-1build1 [14.4 kB] 2024-05-04T12:19:33.610 INFO:teuthology.orchestra.run.smithi052.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 ocfs2-tools amd64 1.8.7-1build1 [646 kB] 2024-05-04T12:19:33.701 INFO:teuthology.orchestra.run.smithi052.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libcap2 amd64 1:2.44-1ubuntu0.22.04.1 [18.3 kB] 2024-05-04T12:19:33.702 INFO:teuthology.orchestra.run.smithi052.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libsqlite3-0 amd64 3.37.2-2ubuntu0.3 [641 kB] 2024-05-04T12:19:33.756 INFO:teuthology.orchestra.run.smithi052.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 exfatprogs amd64 1.1.3-1 [40.3 kB] 2024-05-04T12:19:33.758 INFO:teuthology.orchestra.run.smithi052.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 f2fs-tools amd64 1.14.0-2build1 [169 kB] 2024-05-04T12:19:33.932 INFO:teuthology.orchestra.run.smithi052.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libpmemblk1 amd64 1.11.1-3build1 [65.6 kB] 2024-05-04T12:19:34.225 INFO:teuthology.orchestra.run.smithi052.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 fio amd64 3.28-1 [4,047 kB] 2024-05-04T12:19:34.731 INFO:teuthology.orchestra.run.smithi052.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libcap-dev amd64 1:2.44-1ubuntu0.22.04.1 [39.4 kB] 2024-05-04T12:19:34.732 INFO:teuthology.orchestra.run.smithi052.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgdbm-dev amd64 1.23-1 [117 kB] 2024-05-04T12:19:34.735 INFO:teuthology.orchestra.run.smithi052.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liburing-dev amd64 2.1-2build1 [51.8 kB] 2024-05-04T12:19:34.736 INFO:teuthology.orchestra.run.smithi052.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 sqlite3 amd64 3.37.2-2ubuntu0.3 [768 kB] 2024-05-04T12:19:34.755 INFO:teuthology.orchestra.run.smithi052.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 udftools amd64 2.3-1 [125 kB] 2024-05-04T12:19:35.059 INFO:teuthology.orchestra.run.smithi052.stderr:debconf: unable to initialize frontend: Dialog 2024-05-04T12:19:35.059 INFO:teuthology.orchestra.run.smithi052.stderr:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-05-04T12:19:35.059 INFO:teuthology.orchestra.run.smithi052.stderr:debconf: falling back to frontend: Readline 2024-05-04T12:19:35.069 INFO:teuthology.orchestra.run.smithi052.stderr:debconf: unable to initialize frontend: Readline 2024-05-04T12:19:35.069 INFO:teuthology.orchestra.run.smithi052.stderr:debconf: (This frontend requires a controlling tty.) 2024-05-04T12:19:35.069 INFO:teuthology.orchestra.run.smithi052.stderr:debconf: falling back to frontend: Teletype 2024-05-04T12:19:35.073 INFO:teuthology.orchestra.run.smithi052.stderr:dpkg-preconfigure: unable to re-open stdin: 2024-05-04T12:19:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:Fetched 7,314 kB in 2s (3,861 kB/s) 2024-05-04T12:19:35.413 INFO:teuthology.orchestra.run.smithi052.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176786 files and directories currently installed.) 2024-05-04T12:19:35.419 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../0-uuid-runtime_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-04T12:19:35.552 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking uuid-runtime (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-04T12:19:36.153 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../1-gawk_1%3a5.1.0-1ubuntu0.1_amd64.deb ... 2024-05-04T12:19:36.300 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking gawk (1:5.1.0-1ubuntu0.1) over (1:5.1.0-1build3) ... 2024-05-04T12:19:36.786 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libcorosync-common4:amd64. 2024-05-04T12:19:36.816 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../2-libcorosync-common4_3.1.6-1ubuntu1_amd64.deb ... 2024-05-04T12:19:36.894 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libcorosync-common4:amd64 (3.1.6-1ubuntu1) ... 2024-05-04T12:19:37.190 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libqb100:amd64. 2024-05-04T12:19:37.218 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../3-libqb100_2.0.4-1ubuntu0.2_amd64.deb ... 2024-05-04T12:19:37.256 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libqb100:amd64 (2.0.4-1ubuntu0.2) ... 2024-05-04T12:19:37.601 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libcmap4:amd64. 2024-05-04T12:19:37.630 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../4-libcmap4_3.1.6-1ubuntu1_amd64.deb ... 2024-05-04T12:19:37.667 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libcmap4:amd64 (3.1.6-1ubuntu1) ... 2024-05-04T12:19:38.004 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libdlm3:amd64. 2024-05-04T12:19:38.033 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../5-libdlm3_4.1.1-1build1_amd64.deb ... 2024-05-04T12:19:38.079 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libdlm3:amd64 (4.1.1-1build1) ... 2024-05-04T12:19:38.408 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ocfs2-tools. 2024-05-04T12:19:38.437 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../6-ocfs2-tools_1.8.7-1build1_amd64.deb ... 2024-05-04T12:19:38.466 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ocfs2-tools (1.8.7-1build1) ... 2024-05-04T12:19:38.901 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../7-libcap2_1%3a2.44-1ubuntu0.22.04.1_amd64.deb ... 2024-05-04T12:19:39.039 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libcap2:amd64 (1:2.44-1ubuntu0.22.04.1) over (1:2.44-1build3) ... 2024-05-04T12:19:39.376 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libcap2:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-05-04T12:19:39.850 INFO:teuthology.orchestra.run.smithi052.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176860 files and directories currently installed.) 2024-05-04T12:19:39.856 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../0-libsqlite3-0_3.37.2-2ubuntu0.3_amd64.deb ... 2024-05-04T12:19:39.997 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) over (3.37.2-2ubuntu0.1) ... 2024-05-04T12:19:40.360 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package exfatprogs. 2024-05-04T12:19:40.389 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../1-exfatprogs_1.1.3-1_amd64.deb ... 2024-05-04T12:19:40.426 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking exfatprogs (1.1.3-1) ... 2024-05-04T12:19:40.671 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package f2fs-tools. 2024-05-04T12:19:40.700 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../2-f2fs-tools_1.14.0-2build1_amd64.deb ... 2024-05-04T12:19:40.802 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking f2fs-tools (1.14.0-2build1) ... 2024-05-04T12:19:41.142 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libpmemblk1:amd64. 2024-05-04T12:19:41.171 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../3-libpmemblk1_1.11.1-3build1_amd64.deb ... 2024-05-04T12:19:41.208 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libpmemblk1:amd64 (1.11.1-3build1) ... 2024-05-04T12:19:41.554 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package fio. 2024-05-04T12:19:41.582 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../4-fio_3.28-1_amd64.deb ... 2024-05-04T12:19:41.620 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking fio (3.28-1) ... 2024-05-04T12:19:42.041 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libcap-dev:amd64. 2024-05-04T12:19:42.058 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../5-libcap-dev_1%3a2.44-1ubuntu0.22.04.1_amd64.deb ... 2024-05-04T12:19:42.099 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libcap-dev:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-05-04T12:19:42.329 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libgdbm-dev:amd64. 2024-05-04T12:19:42.346 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../6-libgdbm-dev_1.23-1_amd64.deb ... 2024-05-04T12:19:42.388 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libgdbm-dev:amd64 (1.23-1) ... 2024-05-04T12:19:42.791 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package liburing-dev:amd64. 2024-05-04T12:19:42.820 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../7-liburing-dev_2.1-2build1_amd64.deb ... 2024-05-04T12:19:42.857 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking liburing-dev:amd64 (2.1-2build1) ... 2024-05-04T12:19:43.128 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package sqlite3. 2024-05-04T12:19:43.147 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../8-sqlite3_3.37.2-2ubuntu0.3_amd64.deb ... 2024-05-04T12:19:43.186 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking sqlite3 (3.37.2-2ubuntu0.3) ... 2024-05-04T12:19:43.506 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package udftools. 2024-05-04T12:19:43.535 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../9-udftools_2.3-1_amd64.deb ... 2024-05-04T12:19:43.576 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking udftools (2.3-1) ... 2024-05-04T12:19:43.885 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up exfatprogs (1.1.3-1) ... 2024-05-04T12:19:44.011 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up gawk (1:5.1.0-1ubuntu0.1) ... 2024-05-04T12:19:44.255 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up f2fs-tools (1.14.0-2build1) ... 2024-05-04T12:19:44.381 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) ... 2024-05-04T12:19:44.507 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libqb100:amd64 (2.0.4-1ubuntu0.2) ... 2024-05-04T12:19:44.633 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libcorosync-common4:amd64 (3.1.6-1ubuntu1) ... 2024-05-04T12:19:44.760 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libpmemblk1:amd64 (1.11.1-3build1) ... 2024-05-04T12:19:44.902 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up uuid-runtime (2.37.2-4ubuntu3.4) ... 2024-05-04T12:19:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:uuidd.service is a disabled or a static unit not running, not starting it. 2024-05-04T12:19:45.821 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up udftools (2.3-1) ... 2024-05-04T12:19:45.964 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libdlm3:amd64 (4.1.1-1build1) ... 2024-05-04T12:19:46.090 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libgdbm-dev:amd64 (1.23-1) ... 2024-05-04T12:19:46.216 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libcap-dev:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-05-04T12:19:46.342 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up sqlite3 (3.37.2-2ubuntu0.3) ... 2024-05-04T12:19:46.468 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up liburing-dev:amd64 (2.1-2build1) ... 2024-05-04T12:19:46.594 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libcmap4:amd64 (3.1.6-1ubuntu1) ... 2024-05-04T12:19:46.703 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up fio (3.28-1) ... 2024-05-04T12:19:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ocfs2-tools (1.8.7-1build1) ... 2024-05-04T12:19:47.427 INFO:teuthology.orchestra.run.smithi052.stdout:debconf: unable to initialize frontend: Dialog 2024-05-04T12:19:47.427 INFO:teuthology.orchestra.run.smithi052.stdout:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-05-04T12:19:47.427 INFO:teuthology.orchestra.run.smithi052.stdout:debconf: falling back to frontend: Readline 2024-05-04T12:19:48.151 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for initramfs-tools (0.140ubuntu13.1) ... 2024-05-04T12:19:48.253 INFO:teuthology.orchestra.run.smithi052.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-05-04T12:19:53.528 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:19:53.530 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:20:06.889 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-04T12:20:07.041 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-04T12:20:08.335 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-04T12:20:08.920 DEBUG:teuthology.orchestra.run.smithi052:> sudo useradd -m fsgqa 2024-05-04T12:20:08.930 INFO:teuthology.orchestra.run.smithi052.stderr:useradd: user 'fsgqa' already exists 2024-05-04T12:20:08.931 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-05-04T12:20:08.931 DEBUG:teuthology.orchestra.run.smithi052:> sudo groupadd fsgqa 2024-05-04T12:20:08.984 INFO:teuthology.orchestra.run.smithi052.stderr:groupadd: group 'fsgqa' already exists 2024-05-04T12:20:08.984 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-05-04T12:20:08.985 DEBUG:teuthology.orchestra.run.smithi052:> sudo useradd fsgqa2 2024-05-04T12:20:09.405 DEBUG:teuthology.orchestra.run.smithi052:> sudo useradd 123456-fsgqa 2024-05-04T12:20:09.793 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-05-04T12:20:09.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4b3af640 1 Processor -- start 2024-05-04T12:20:09.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4b3af640 1 -- start start 2024-05-04T12:20:09.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4b3af640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9b44146160 0x7f9b440ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:20:09.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4b3af640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9b440cfb60 0x7f9b4406ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:20:09.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4b3af640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b4406f390 0x7f9b44160310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:20:09.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4b3af640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f9b44146540 con 0x7f9b440cfb60 2024-05-04T12:20:09.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4b3af640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f9b440cd140 con 0x7f9b44146160 2024-05-04T12:20:09.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4b3af640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f9b440d0e10 con 0x7f9b4406f390 2024-05-04T12:20:09.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4894a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9b440cfb60 0x7f9b4406ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:20:09.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4894a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9b440cfb60 0x7f9b4406ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42226/0 (socket says 172.21.15.52:42226) 2024-05-04T12:20:09.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4894a640 1 -- 172.21.15.52:0/2245873622 learned_addr learned my addr 172.21.15.52:0/2245873622 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:20:09.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4914b640 1 --2- 172.21.15.52:0/2245873622 >> v2:172.21.15.111:3300/0 conn(0x7f9b44146160 0x7f9b440ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:20:09.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4994c640 1 --2- 172.21.15.52:0/2245873622 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b4406f390 0x7f9b44160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:20:09.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4894a640 1 -- 172.21.15.52:0/2245873622 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b4406f390 msgr2=0x7f9b44160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:20:09.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4894a640 1 --2- 172.21.15.52:0/2245873622 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b4406f390 0x7f9b44160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:09.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4894a640 1 -- 172.21.15.52:0/2245873622 >> v2:172.21.15.111:3300/0 conn(0x7f9b44146160 msgr2=0x7f9b440ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:20:09.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4894a640 1 --2- 172.21.15.52:0/2245873622 >> v2:172.21.15.111:3300/0 conn(0x7f9b44146160 0x7f9b440ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:09.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.920+0000 7f9b4894a640 1 -- 172.21.15.52:0/2245873622 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b440d10f0 con 0x7f9b440cfb60 2024-05-04T12:20:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b4894a640 1 --2- 172.21.15.52:0/2245873622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b440cfb60 0x7f9b4406ee50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9b4406eb00 tx=0x7f9b3801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f48c79b692fffd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:20:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b43fff640 1 -- 172.21.15.52:0/2245873622 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b380a9430 con 0x7f9b440cfb60 2024-05-04T12:20:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b43fff640 1 -- 172.21.15.52:0/2245873622 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b380ae070 con 0x7f9b440cfb60 2024-05-04T12:20:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b43fff640 1 -- 172.21.15.52:0/2245873622 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b380a97f0 con 0x7f9b440cfb60 2024-05-04T12:20:09.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b4b3af640 1 -- 172.21.15.52:0/2245873622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b440cfb60 msgr2=0x7f9b4406ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:20:09.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b4b3af640 1 --2- 172.21.15.52:0/2245873622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b440cfb60 0x7f9b4406ee50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9b4406eb00 tx=0x7f9b3801c6a0 comp rx=0 tx=0).stop 2024-05-04T12:20:09.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b4b3af640 1 -- 172.21.15.52:0/2245873622 shutdown_connections 2024-05-04T12:20:09.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b4b3af640 1 --2- 172.21.15.52:0/2245873622 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b4406f390 0x7f9b44160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:09.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b4b3af640 1 --2- 172.21.15.52:0/2245873622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b440cfb60 0x7f9b4406ee50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:09.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b4b3af640 1 --2- 172.21.15.52:0/2245873622 >> v2:172.21.15.111:3300/0 conn(0x7f9b44146160 0x7f9b440ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:09.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b4b3af640 1 -- 172.21.15.52:0/2245873622 >> 172.21.15.52:0/2245873622 conn(0x7f9b4406b210 msgr2=0x7f9b4406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:20:09.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b4b3af640 1 -- 172.21.15.52:0/2245873622 shutdown_connections 2024-05-04T12:20:09.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b4b3af640 1 -- 172.21.15.52:0/2245873622 wait complete. 2024-05-04T12:20:09.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b4b3af640 1 Processor -- start 2024-05-04T12:20:09.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.924+0000 7f9b4b3af640 1 -- start start 2024-05-04T12:20:09.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4b3af640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b4406f390 0x7f9b44137870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:20:09.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4b3af640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b440cfb60 0x7f9b44137e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:20:09.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4b3af640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b44146160 0x7f9b44138350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:20:09.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4b3af640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9b440d1a70 con 0x7f9b44146160 2024-05-04T12:20:09.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4b3af640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9b440d18f0 con 0x7f9b4406f390 2024-05-04T12:20:09.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4b3af640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f9b440d1bf0 con 0x7f9b440cfb60 2024-05-04T12:20:09.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4994c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b44146160 0x7f9b44138350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4994c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b44146160 0x7f9b44138350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42228/0 (socket says 172.21.15.52:42228) 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4994c640 1 -- 172.21.15.52:0/3111655974 learned_addr learned my addr 172.21.15.52:0/3111655974 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4894a640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b440cfb60 0x7f9b44137e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4914b640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b4406f390 0x7f9b44137870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4994c640 1 -- 172.21.15.52:0/3111655974 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b440cfb60 msgr2=0x7f9b44137e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4994c640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b440cfb60 0x7f9b44137e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4994c640 1 -- 172.21.15.52:0/3111655974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b4406f390 msgr2=0x7f9b44137870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4994c640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b4406f390 0x7f9b44137870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4994c640 1 -- 172.21.15.52:0/3111655974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4413cdb0 con 0x7f9b44146160 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4914b640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b4406f390 0x7f9b44137870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4994c640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b44146160 0x7f9b44138350 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00d830 tx=0x7f9b3c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b41ffb640 1 -- 172.21.15.52:0/3111655974 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c018020 con 0x7f9b44146160 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b41ffb640 1 -- 172.21.15.52:0/3111655974 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b3c009660 con 0x7f9b44146160 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b41ffb640 1 -- 172.21.15.52:0/3111655974 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c009a50 con 0x7f9b44146160 2024-05-04T12:20:09.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4b3af640 1 -- 172.21.15.52:0/3111655974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4413f710 con 0x7f9b44146160 2024-05-04T12:20:09.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4b3af640 1 -- 172.21.15.52:0/3111655974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b44160140 con 0x7f9b44146160 2024-05-04T12:20:09.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4b3af640 1 -- 172.21.15.52:0/3111655974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b44062e50 con 0x7f9b44146160 2024-05-04T12:20:09.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b41ffb640 1 -- 172.21.15.52:0/3111655974 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9b3c01b020 con 0x7f9b44146160 2024-05-04T12:20:09.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b41ffb640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f9b300678c0 0x7f9b30069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:20:09.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b41ffb640 1 -- 172.21.15.52:0/3111655974 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f9b3c086340 con 0x7f9b44146160 2024-05-04T12:20:09.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4914b640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f9b300678c0 0x7f9b30069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:20:09.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.928+0000 7f9b4914b640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f9b300678c0 0x7f9b30069d80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9b340097a0 tx=0x7f9b34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:20:09.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:09.932+0000 7f9b41ffb640 1 -- 172.21.15.52:0/3111655974 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9b3c022020 con 0x7f9b44146160 2024-05-04T12:20:10.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.160+0000 7f9b4b3af640 1 -- 172.21.15.52:0/3111655974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9b440cf4d0 con 0x7f9b44146160 2024-05-04T12:20:10.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.160+0000 7f9b41ffb640 1 -- 172.21.15.52:0/3111655974 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1285 (secure 0 0 0) 0x7f9b3c011340 con 0x7f9b44146160 2024-05-04T12:20:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:20:10.167 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":1,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","modified":"2024-05-04T12:17:24.735371Z","created":"2024-05-04T12:17:24.735371Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3301","nonce":0},{"type":"v1","addr":"172.21.15.111:6790","nonce":0}]},"addr":"172.21.15.111:6790/0","public_addr":"172.21.15.111:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-04T12:20:10.167 INFO:teuthology.orchestra.run.smithi052.stderr:dumped monmap epoch 1 2024-05-04T12:20:10.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.164+0000 7f9b4b3af640 1 -- 172.21.15.52:0/3111655974 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f9b300678c0 msgr2=0x7f9b30069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:20:10.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.164+0000 7f9b4b3af640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f9b300678c0 0x7f9b30069d80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9b340097a0 tx=0x7f9b34008040 comp rx=0 tx=0).stop 2024-05-04T12:20:10.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.164+0000 7f9b4b3af640 1 -- 172.21.15.52:0/3111655974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b44146160 msgr2=0x7f9b44138350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:20:10.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.164+0000 7f9b4b3af640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b44146160 0x7f9b44138350 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00d830 tx=0x7f9b3c00dcf0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.164+0000 7f9b4b3af640 1 -- 172.21.15.52:0/3111655974 shutdown_connections 2024-05-04T12:20:10.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.164+0000 7f9b4b3af640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f9b300678c0 0x7f9b30069d80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.164+0000 7f9b4b3af640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b44146160 0x7f9b44138350 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.164+0000 7f9b4b3af640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b440cfb60 0x7f9b44137e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.164+0000 7f9b4b3af640 1 --2- 172.21.15.52:0/3111655974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b4406f390 0x7f9b44137870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.164+0000 7f9b4b3af640 1 -- 172.21.15.52:0/3111655974 >> 172.21.15.52:0/3111655974 conn(0x7f9b4406b210 msgr2=0x7f9b440b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:20:10.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.164+0000 7f9b4b3af640 1 -- 172.21.15.52:0/3111655974 shutdown_connections 2024-05-04T12:20:10.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.164+0000 7f9b4b3af640 1 -- 172.21.15.52:0/3111655974 wait complete. 2024-05-04T12:20:10.187 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2024-05-04T12:20:10.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0ff4ff1640 1 Processor -- start 2024-05-04T12:20:10.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0ff4ff1640 1 -- start start 2024-05-04T12:20:10.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0ff4ff1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0ff015e240 0x7f0ff006d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:20:10.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0ff4ff1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0ff00ae250 0x7f0ff00b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:20:10.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0ff4ff1640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ff00ae6b0 0x7f0ff00b5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:20:10.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0ff4ff1640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f0ff00ceb10 con 0x7f0ff015e240 2024-05-04T12:20:10.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0ff4ff1640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f0ff00adc50 con 0x7f0ff00ae250 2024-05-04T12:20:10.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0ff4ff1640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff00ced80 con 0x7f0ff00ae6b0 2024-05-04T12:20:10.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0fee59c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0ff015e240 0x7f0ff006d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:20:10.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0fee59c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0ff015e240 0x7f0ff006d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42230/0 (socket says 172.21.15.52:42230) 2024-05-04T12:20:10.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0fee59c640 1 -- 172.21.15.52:0/4249210139 learned_addr learned my addr 172.21.15.52:0/4249210139 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:20:10.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0fedd9b640 1 --2- 172.21.15.52:0/4249210139 >> v2:172.21.15.111:3300/0 conn(0x7f0ff00ae250 0x7f0ff00b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:20:10.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0feed9d640 1 --2- 172.21.15.52:0/4249210139 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ff00ae6b0 0x7f0ff00b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:20:10.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0fee59c640 1 -- 172.21.15.52:0/4249210139 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ff00ae6b0 msgr2=0x7f0ff00b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:20:10.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0fee59c640 1 --2- 172.21.15.52:0/4249210139 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ff00ae6b0 0x7f0ff00b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0fee59c640 1 -- 172.21.15.52:0/4249210139 >> v2:172.21.15.111:3300/0 conn(0x7f0ff00ae250 msgr2=0x7f0ff00b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T12:20:10.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0fee59c640 1 --2- 172.21.15.52:0/4249210139 >> v2:172.21.15.111:3300/0 conn(0x7f0ff00ae250 0x7f0ff00b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0fee59c640 1 -- 172.21.15.52:0/4249210139 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff00cf060 con 0x7f0ff015e240 2024-05-04T12:20:10.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.312+0000 7f0fedd9b640 1 --2- 172.21.15.52:0/4249210139 >> v2:172.21.15.111:3300/0 conn(0x7f0ff00ae250 0x7f0ff00b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T12:20:10.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.316+0000 7f0fee59c640 1 --2- 172.21.15.52:0/4249210139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ff015e240 0x7f0ff006d730 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00b40f0 tx=0x7f0fd801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a94b48ba78928c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:20:10.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.316+0000 7f0fed59a640 1 -- 172.21.15.52:0/4249210139 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd80b0070 con 0x7f0ff015e240 2024-05-04T12:20:10.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.316+0000 7f0fed59a640 1 -- 172.21.15.52:0/4249210139 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd80ab600 con 0x7f0ff015e240 2024-05-04T12:20:10.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.316+0000 7f0fed59a640 1 -- 172.21.15.52:0/4249210139 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd80ab9d0 con 0x7f0ff015e240 2024-05-04T12:20:10.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.316+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4249210139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ff015e240 msgr2=0x7f0ff006d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:20:10.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.316+0000 7f0ff4ff1640 1 --2- 172.21.15.52:0/4249210139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ff015e240 0x7f0ff006d730 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00b40f0 tx=0x7f0fd801c520 comp rx=0 tx=0).stop 2024-05-04T12:20:10.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.316+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4249210139 shutdown_connections 2024-05-04T12:20:10.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.316+0000 7f0ff4ff1640 1 --2- 172.21.15.52:0/4249210139 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ff00ae6b0 0x7f0ff00b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.316+0000 7f0ff4ff1640 1 --2- 172.21.15.52:0/4249210139 >> v2:172.21.15.111:3300/0 conn(0x7f0ff00ae250 0x7f0ff00b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.316+0000 7f0ff4ff1640 1 --2- 172.21.15.52:0/4249210139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ff015e240 0x7f0ff006d730 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.316+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4249210139 >> 172.21.15.52:0/4249210139 conn(0x7f0ff00b3190 msgr2=0x7f0ff00b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:20:10.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.316+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4249210139 shutdown_connections 2024-05-04T12:20:10.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.316+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4249210139 wait complete. 2024-05-04T12:20:10.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0ff4ff1640 1 Processor -- start 2024-05-04T12:20:10.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0ff4ff1640 1 -- start start 2024-05-04T12:20:10.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0ff4ff1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ff00ae250 0x7f0ff013e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:20:10.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0ff4ff1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ff00ae6b0 0x7f0ff0071a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:20:10.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0ff4ff1640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ff015e240 0x7f0ff0153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:20:10.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0ff4ff1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff00cff10 con 0x7f0ff00ae250 2024-05-04T12:20:10.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0ff4ff1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff00cfd90 con 0x7f0ff00ae6b0 2024-05-04T12:20:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0ff4ff1640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff00d0090 con 0x7f0ff015e240 2024-05-04T12:20:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fee59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ff00ae250 0x7f0ff013e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:20:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fee59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ff00ae250 0x7f0ff013e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42244/0 (socket says 172.21.15.52:42244) 2024-05-04T12:20:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fee59c640 1 -- 172.21.15.52:0/4003467164 learned_addr learned my addr 172.21.15.52:0/4003467164 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T12:20:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fedd9b640 1 --2- 172.21.15.52:0/4003467164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ff00ae6b0 0x7f0ff0071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:20:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0feed9d640 1 --2- 172.21.15.52:0/4003467164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ff015e240 0x7f0ff0153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:20:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fee59c640 1 -- 172.21.15.52:0/4003467164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ff015e240 msgr2=0x7f0ff0153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:20:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fee59c640 1 --2- 172.21.15.52:0/4003467164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ff015e240 0x7f0ff0153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fee59c640 1 -- 172.21.15.52:0/4003467164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ff00ae6b0 msgr2=0x7f0ff0071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:20:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fee59c640 1 --2- 172.21.15.52:0/4003467164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ff00ae6b0 0x7f0ff0071a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fee59c640 1 -- 172.21.15.52:0/4003467164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff0072350 con 0x7f0ff00ae250 2024-05-04T12:20:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fee59c640 1 --2- 172.21.15.52:0/4003467164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ff00ae250 0x7f0ff013e260 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80ae040 tx=0x7f0fd801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:20:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fe6ffd640 1 -- 172.21.15.52:0/4003467164 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd80ba020 con 0x7f0ff00ae250 2024-05-04T12:20:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fe6ffd640 1 -- 172.21.15.52:0/4003467164 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd80b6420 con 0x7f0ff00ae250 2024-05-04T12:20:10.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4003467164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff0072630 con 0x7f0ff00ae250 2024-05-04T12:20:10.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fe6ffd640 1 -- 172.21.15.52:0/4003467164 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd80b0040 con 0x7f0ff00ae250 2024-05-04T12:20:10.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4003467164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff0072bf0 con 0x7f0ff00ae250 2024-05-04T12:20:10.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4003467164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff0062e50 con 0x7f0ff00ae250 2024-05-04T12:20:10.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fe6ffd640 1 -- 172.21.15.52:0/4003467164 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0fd80bd020 con 0x7f0ff00ae250 2024-05-04T12:20:10.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.320+0000 7f0fe6ffd640 1 --2- 172.21.15.52:0/4003467164 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0fd00678c0 0x7f0fd0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T12:20:10.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.324+0000 7f0fe6ffd640 1 -- 172.21.15.52:0/4003467164 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0fd811fd90 con 0x7f0ff00ae250 2024-05-04T12:20:10.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.324+0000 7f0fedd9b640 1 --2- 172.21.15.52:0/4003467164 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0fd00678c0 0x7f0fd0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T12:20:10.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.324+0000 7f0fedd9b640 1 --2- 172.21.15.52:0/4003467164 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0fd00678c0 0x7f0fd0069d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0ff015ad30 tx=0x7f0fe00039a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T12:20:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.328+0000 7f0fe6ffd640 1 -- 172.21.15.52:0/4003467164 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0fd80c4030 con 0x7f0ff00ae250 2024-05-04T12:20:10.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.544+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4003467164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) -- 0x7f0ff006f230 con 0x7f0ff00ae250 2024-05-04T12:20:10.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.544+0000 7f0fe6ffd640 1 -- 172.21.15.52:0/4003467164 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v15) ==== 82+0+151 (secure 0 0 0) 0x7f0fd8016110 con 0x7f0ff00ae250 2024-05-04T12:20:10.548 INFO:teuthology.orchestra.run.smithi052.stdout:[client.admin] 2024-05-04T12:20:10.548 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBUJzZmagRtORAAnh8L3cPjjG9dyD+slAZ8Ow== 2024-05-04T12:20:10.548 INFO:teuthology.orchestra.run.smithi052.stdout: caps mds = "allow *" 2024-05-04T12:20:10.548 INFO:teuthology.orchestra.run.smithi052.stdout: caps mgr = "allow *" 2024-05-04T12:20:10.548 INFO:teuthology.orchestra.run.smithi052.stdout: caps mon = "allow *" 2024-05-04T12:20:10.548 INFO:teuthology.orchestra.run.smithi052.stdout: caps osd = "allow *" 2024-05-04T12:20:10.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.544+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4003467164 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0fd00678c0 msgr2=0x7f0fd0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:20:10.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.544+0000 7f0ff4ff1640 1 --2- 172.21.15.52:0/4003467164 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0fd00678c0 0x7f0fd0069d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0ff015ad30 tx=0x7f0fe00039a0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.544+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4003467164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ff00ae250 msgr2=0x7f0ff013e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T12:20:10.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.544+0000 7f0ff4ff1640 1 --2- 172.21.15.52:0/4003467164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ff00ae250 0x7f0ff013e260 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80ae040 tx=0x7f0fd801ad60 comp rx=0 tx=0).stop 2024-05-04T12:20:10.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.544+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4003467164 shutdown_connections 2024-05-04T12:20:10.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.544+0000 7f0ff4ff1640 1 --2- 172.21.15.52:0/4003467164 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0fd00678c0 0x7f0fd0069d80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.544+0000 7f0ff4ff1640 1 --2- 172.21.15.52:0/4003467164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ff015e240 0x7f0ff0153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.544+0000 7f0ff4ff1640 1 --2- 172.21.15.52:0/4003467164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ff00ae6b0 0x7f0ff0071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.544+0000 7f0ff4ff1640 1 --2- 172.21.15.52:0/4003467164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ff00ae250 0x7f0ff013e260 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T12:20:10.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.544+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4003467164 >> 172.21.15.52:0/4003467164 conn(0x7f0ff00b3190 msgr2=0x7f0ff00b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T12:20:10.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.544+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4003467164 shutdown_connections 2024-05-04T12:20:10.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T12:20:10.544+0000 7f0ff4ff1640 1 -- 172.21.15.52:0/4003467164 wait complete. 2024-05-04T12:20:10.566 DEBUG:teuthology.orchestra.run.smithi052:> mktemp 2024-05-04T12:20:10.571 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.1tS441WIWC 2024-05-04T12:20:10.571 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:20:10.571 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/tmp/tmp.1tS441WIWC 2024-05-04T12:20:10.620 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:20:10.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/tmp/tmp.BtfdD8Y1eWxfstests-dev/local.config 2024-05-04T12:20:10.675 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.FGpMFsgr0Qtest export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.JrqTWk0bCcscratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.52:6789 -k /tmp/tmp.1tS441WIWC --client_mountpoint /test" 2024-05-04T12:20:10.676 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T12:20:10.676 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/tmp/tmp.BtfdD8Y1eWxfstests-dev/ceph.exclude 2024-05-04T12:20:10.735 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo make) 2024-05-04T12:20:10.795 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize -cfi 2024-05-04T12:20:11.072 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: putting auxiliary files in '.'. 2024-05-04T12:20:11.073 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file './config.guess' 2024-05-04T12:20:11.078 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file './config.sub' 2024-05-04T12:20:11.090 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file './install-sh' 2024-05-04T12:20:11.096 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file './ltmain.sh' 2024-05-04T12:20:11.120 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-05-04T12:20:11.120 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-05-04T12:20:11.127 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-05-04T12:20:11.133 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-05-04T12:20:11.138 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-05-04T12:20:11.144 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-05-04T12:20:11.149 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-05-04T12:20:11.151 INFO:teuthology.orchestra.run.smithi052.stdout:cp include/install-sh . 2024-05-04T12:20:11.152 INFO:teuthology.orchestra.run.smithi052.stdout:aclocal -I m4 2024-05-04T12:20:12.319 INFO:teuthology.orchestra.run.smithi052.stdout:autoheader 2024-05-04T12:20:12.689 INFO:teuthology.orchestra.run.smithi052.stdout:autoconf 2024-05-04T12:20:13.087 INFO:teuthology.orchestra.run.smithi052.stdout:./configure \ 2024-05-04T12:20:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: --libexecdir=/usr/lib \ 2024-05-04T12:20:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: --exec_prefix=/var/lib 2024-05-04T12:20:13.539 INFO:teuthology.orchestra.run.smithi052.stdout:checking build system type... x86_64-pc-linux-gnu 2024-05-04T12:20:13.539 INFO:teuthology.orchestra.run.smithi052.stdout:checking host system type... x86_64-pc-linux-gnu 2024-05-04T12:20:13.582 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to print strings... printf 2024-05-04T12:20:13.582 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gcc... gcc 2024-05-04T12:20:13.761 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether the C compiler works... yes 2024-05-04T12:20:13.761 INFO:teuthology.orchestra.run.smithi052.stdout:checking for C compiler default output file name... a.out 2024-05-04T12:20:13.801 INFO:teuthology.orchestra.run.smithi052.stdout:checking for suffix of executables... 2024-05-04T12:20:13.864 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether we are cross compiling... no 2024-05-04T12:20:13.882 INFO:teuthology.orchestra.run.smithi052.stdout:checking for suffix of object files... o 2024-05-04T12:20:13.901 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether the compiler supports GNU C... yes 2024-05-04T12:20:13.917 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether gcc accepts -g... yes 2024-05-04T12:20:14.092 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gcc option to enable C11 features... none needed 2024-05-04T12:20:14.097 INFO:teuthology.orchestra.run.smithi052.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-05-04T12:20:14.101 INFO:teuthology.orchestra.run.smithi052.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-04T12:20:14.103 INFO:teuthology.orchestra.run.smithi052.stdout:checking for egrep... /usr/bin/grep -E 2024-05-04T12:20:14.105 INFO:teuthology.orchestra.run.smithi052.stdout:checking for fgrep... /usr/bin/grep -F 2024-05-04T12:20:14.111 INFO:teuthology.orchestra.run.smithi052.stdout:checking for ld used by gcc... /usr/bin/ld 2024-05-04T12:20:14.113 INFO:teuthology.orchestra.run.smithi052.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-05-04T12:20:14.147 INFO:teuthology.orchestra.run.smithi052.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-05-04T12:20:14.170 INFO:teuthology.orchestra.run.smithi052.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-05-04T12:20:14.171 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether ln -s works... yes 2024-05-04T12:20:14.180 INFO:teuthology.orchestra.run.smithi052.stdout:checking the maximum length of command line arguments... 1572864 2024-05-04T12:20:14.184 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-05-04T12:20:14.184 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-05-04T12:20:14.184 INFO:teuthology.orchestra.run.smithi052.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-05-04T12:20:14.184 INFO:teuthology.orchestra.run.smithi052.stdout:checking for objdump... objdump 2024-05-04T12:20:14.185 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to recognize dependent libraries... pass_all 2024-05-04T12:20:14.185 INFO:teuthology.orchestra.run.smithi052.stdout:checking for dlltool... no 2024-05-04T12:20:14.185 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-05-04T12:20:14.185 INFO:teuthology.orchestra.run.smithi052.stdout:checking for ar... ar 2024-05-04T12:20:14.228 INFO:teuthology.orchestra.run.smithi052.stdout:checking for archiver @FILE support... @ 2024-05-04T12:20:14.228 INFO:teuthology.orchestra.run.smithi052.stdout:checking for strip... strip 2024-05-04T12:20:14.229 INFO:teuthology.orchestra.run.smithi052.stdout:checking for ranlib... ranlib 2024-05-04T12:20:14.229 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gawk... gawk 2024-05-04T12:20:14.298 INFO:teuthology.orchestra.run.smithi052.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-05-04T12:20:14.299 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sysroot... no 2024-05-04T12:20:14.304 INFO:teuthology.orchestra.run.smithi052.stdout:checking for a working dd... /usr/bin/dd 2024-05-04T12:20:14.308 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-05-04T12:20:14.322 INFO:teuthology.orchestra.run.smithi052.stdout:checking for mt... mt 2024-05-04T12:20:14.337 INFO:teuthology.orchestra.run.smithi052.stdout:checking if mt is a manifest tool... no 2024-05-04T12:20:14.368 INFO:teuthology.orchestra.run.smithi052.stdout:checking for stdio.h... yes 2024-05-04T12:20:14.443 INFO:teuthology.orchestra.run.smithi052.stdout:checking for stdlib.h... yes 2024-05-04T12:20:14.478 INFO:teuthology.orchestra.run.smithi052.stdout:checking for string.h... yes 2024-05-04T12:20:14.535 INFO:teuthology.orchestra.run.smithi052.stdout:checking for inttypes.h... yes 2024-05-04T12:20:14.563 INFO:teuthology.orchestra.run.smithi052.stdout:checking for stdint.h... yes 2024-05-04T12:20:14.593 INFO:teuthology.orchestra.run.smithi052.stdout:checking for strings.h... yes 2024-05-04T12:20:14.634 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/stat.h... yes 2024-05-04T12:20:14.664 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/types.h... yes 2024-05-04T12:20:14.715 INFO:teuthology.orchestra.run.smithi052.stdout:checking for unistd.h... yes 2024-05-04T12:20:14.768 INFO:teuthology.orchestra.run.smithi052.stdout:checking for dlfcn.h... yes 2024-05-04T12:20:14.772 INFO:teuthology.orchestra.run.smithi052.stdout:checking for objdir... .libs 2024-05-04T12:20:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-05-04T12:20:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-05-04T12:20:14.882 INFO:teuthology.orchestra.run.smithi052.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-05-04T12:20:15.089 INFO:teuthology.orchestra.run.smithi052.stdout:checking if gcc static flag -static works... yes 2024-05-04T12:20:15.120 INFO:teuthology.orchestra.run.smithi052.stdout:checking if gcc supports -c -o file.o... yes 2024-05-04T12:20:15.120 INFO:teuthology.orchestra.run.smithi052.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-05-04T12:20:15.130 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-05-04T12:20:15.159 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether -lc should be explicitly linked in... no 2024-05-04T12:20:15.239 INFO:teuthology.orchestra.run.smithi052.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-05-04T12:20:15.239 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to hardcode library paths into programs... immediate 2024-05-04T12:20:15.251 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether stripping libraries is possible... yes 2024-05-04T12:20:15.251 INFO:teuthology.orchestra.run.smithi052.stdout:checking if libtool supports shared libraries... yes 2024-05-04T12:20:15.251 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether to build shared libraries... yes 2024-05-04T12:20:15.251 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether to build static libraries... yes 2024-05-04T12:20:15.262 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gcc... (cached) gcc 2024-05-04T12:20:15.291 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether the compiler supports GNU C... (cached) yes 2024-05-04T12:20:15.291 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-04T12:20:15.292 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gcc option to enable C11 features... (cached) none needed 2024-05-04T12:20:15.292 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gmake... /usr/bin/gmake 2024-05-04T12:20:15.292 INFO:teuthology.orchestra.run.smithi052.stderr:./configure: line 12753: test: $(SHELL): binary operator expected 2024-05-04T12:20:15.293 INFO:teuthology.orchestra.run.smithi052.stdout:checking for tar... /bin/tar 2024-05-04T12:20:15.293 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gzip... /bin/gzip 2024-05-04T12:20:15.304 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether gcc -MM is supported... yes 2024-05-04T12:20:15.304 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sort... /bin/sort 2024-05-04T12:20:15.304 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether ln -s works... yes 2024-05-04T12:20:15.305 INFO:teuthology.orchestra.run.smithi052.stdout:checking for rpm... no 2024-05-04T12:20:15.342 INFO:teuthology.orchestra.run.smithi052.stdout:checking for assert.h... yes 2024-05-04T12:20:15.384 INFO:teuthology.orchestra.run.smithi052.stdout:checking for bstring.h... no 2024-05-04T12:20:15.421 INFO:teuthology.orchestra.run.smithi052.stdout:checking for libgen.h... yes 2024-05-04T12:20:15.488 INFO:teuthology.orchestra.run.smithi052.stdout:checking for dirent.h... yes 2024-05-04T12:20:15.562 INFO:teuthology.orchestra.run.smithi052.stdout:checking for errno.h... yes 2024-05-04T12:20:15.597 INFO:teuthology.orchestra.run.smithi052.stdout:checking for malloc.h... yes 2024-05-04T12:20:15.631 INFO:teuthology.orchestra.run.smithi052.stdout:checking for uuid.h... no 2024-05-04T12:20:15.674 INFO:teuthology.orchestra.run.smithi052.stdout:checking for uuid/uuid.h... yes 2024-05-04T12:20:15.716 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/uuid.h... no 2024-05-04T12:20:15.754 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/file.h... yes 2024-05-04T12:20:15.790 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/fcntl.h... yes 2024-05-04T12:20:15.823 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/syssgi.h... no 2024-05-04T12:20:15.871 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/param.h... yes 2024-05-04T12:20:15.872 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/stat.h... (cached) yes 2024-05-04T12:20:15.911 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/statvfs.h... yes 2024-05-04T12:20:15.948 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/time.h... yes 2024-05-04T12:20:15.995 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/ioctl.h... yes 2024-05-04T12:20:16.032 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/wait.h... yes 2024-05-04T12:20:16.032 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/types.h... (cached) yes 2024-05-04T12:20:16.032 INFO:teuthology.orchestra.run.smithi052.stdout:checking for strings.h... (cached) yes 2024-05-04T12:20:16.065 INFO:teuthology.orchestra.run.smithi052.stdout:checking for err.h... yes 2024-05-04T12:20:16.098 INFO:teuthology.orchestra.run.smithi052.stdout:checking for linux/falloc.h... yes 2024-05-04T12:20:16.132 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/fs/xfs_fsops.h... no 2024-05-04T12:20:16.165 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/fs/xfs_itable.h... no 2024-05-04T12:20:16.199 INFO:teuthology.orchestra.run.smithi052.stdout:checking for xfs/platform_defs.h... no 2024-05-04T12:20:16.234 INFO:teuthology.orchestra.run.smithi052.stdout:checking for btrfs/ioctl.h... no 2024-05-04T12:20:16.271 INFO:teuthology.orchestra.run.smithi052.stdout:checking for cifs/ioctl.h... no 2024-05-04T12:20:16.314 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/mman.h... yes 2024-05-04T12:20:16.348 INFO:teuthology.orchestra.run.smithi052.stdout:checking for linux/ext4.h... no 2024-05-04T12:20:16.362 INFO:teuthology.orchestra.run.smithi052.stdout:checking for xfs/xfs_log_format.h... no 2024-05-04T12:20:16.406 INFO:teuthology.orchestra.run.smithi052.stdout:checking for uuid_compare... no 2024-05-04T12:20:16.441 INFO:teuthology.orchestra.run.smithi052.stdout:checking for uuid_compare in -luuid... yes 2024-05-04T12:20:16.576 INFO:teuthology.orchestra.run.smithi052.stdout:checking for xfs/xfs.h... yes 2024-05-04T12:20:16.595 INFO:teuthology.orchestra.run.smithi052.stdout:checking for xfs/libxfs.h... no 2024-05-04T12:20:16.635 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gcc options needed to detect all undeclared functions... none needed 2024-05-04T12:20:16.655 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether xlog_assign_lsn is declared... no 2024-05-04T12:20:16.698 INFO:teuthology.orchestra.run.smithi052.stdout:checking for xfs/xqm.h... yes 2024-05-04T12:20:16.759 INFO:teuthology.orchestra.run.smithi052.stdout:checking xfsctl from xfs/xfs.h... ok 2024-05-04T12:20:16.797 INFO:teuthology.orchestra.run.smithi052.stdout:checking for xfs/handle.h... yes 2024-05-04T12:20:16.834 INFO:teuthology.orchestra.run.smithi052.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-05-04T12:20:16.873 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/acl.h... yes 2024-05-04T12:20:16.915 INFO:teuthology.orchestra.run.smithi052.stdout:checking for acl/libacl.h... yes 2024-05-04T12:20:16.951 INFO:teuthology.orchestra.run.smithi052.stdout:checking for acl_init in -lacl... yes 2024-05-04T12:20:16.982 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gdbm-ndbm.h... no 2024-05-04T12:20:17.013 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gdbm/ndbm.h... no 2024-05-04T12:20:17.044 INFO:teuthology.orchestra.run.smithi052.stdout:checking for ndbm.h... no 2024-05-04T12:20:17.077 INFO:teuthology.orchestra.run.smithi052.stdout:checking for libaio.h... yes 2024-05-04T12:20:17.132 INFO:teuthology.orchestra.run.smithi052.stdout:checking for liburing.h... yes 2024-05-04T12:20:17.163 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/capability.h... yes 2024-05-04T12:20:17.194 INFO:teuthology.orchestra.run.smithi052.stdout:checking for linux/fiemap.h... yes 2024-05-04T12:20:17.229 INFO:teuthology.orchestra.run.smithi052.stdout:checking for fallocate... yes 2024-05-04T12:20:17.271 INFO:teuthology.orchestra.run.smithi052.stdout:checking for open_by_handle_at... yes 2024-05-04T12:20:17.303 INFO:teuthology.orchestra.run.smithi052.stdout:checking for linux/fs.h... yes 2024-05-04T12:20:17.340 INFO:teuthology.orchestra.run.smithi052.stdout:checking for btrfsutil.h... no 2024-05-04T12:20:17.390 INFO:teuthology.orchestra.run.smithi052.stdout:checking for copy_file_range... yes 2024-05-04T12:20:17.440 INFO:teuthology.orchestra.run.smithi052.stdout:checking for SEEK_DATA... yes 2024-05-04T12:20:17.503 INFO:teuthology.orchestra.run.smithi052.stdout:checking for BMV_OF_SHARED... yes 2024-05-04T12:20:17.543 INFO:teuthology.orchestra.run.smithi052.stdout:checking for nftw... yes 2024-05-04T12:20:17.588 INFO:teuthology.orchestra.run.smithi052.stdout:checking for RLIMIT_NOFILE... yes 2024-05-04T12:20:17.632 INFO:teuthology.orchestra.run.smithi052.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-05-04T12:20:17.670 INFO:teuthology.orchestra.run.smithi052.stdout:checking for FICLONE... yes 2024-05-04T12:20:17.706 INFO:teuthology.orchestra.run.smithi052.stdout:checking for renameat2... yes 2024-05-04T12:20:17.742 INFO:teuthology.orchestra.run.smithi052.stdout:checking for reallocarray... yes 2024-05-04T12:20:17.774 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct mount_attr... yes 2024-05-04T12:20:17.810 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_qgroup_limit... yes 2024-05-04T12:20:17.847 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-05-04T12:20:17.885 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-05-04T12:20:17.924 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-05-04T12:20:17.962 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-05-04T12:20:17.998 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-05-04T12:20:18.038 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-05-04T12:20:18.075 INFO:teuthology.orchestra.run.smithi052.stdout:checking for linux/btrfs.h... yes 2024-05-04T12:20:18.111 INFO:teuthology.orchestra.run.smithi052.stdout:checking for linux/btrfs_tree.h... yes 2024-05-04T12:20:18.133 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-05-04T12:20:18.167 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-05-04T12:20:18.185 INFO:teuthology.orchestra.run.smithi052.stdout:configure: creating ./config.status 2024-05-04T12:20:18.612 INFO:teuthology.orchestra.run.smithi052.stdout:config.status: creating include/builddefs 2024-05-04T12:20:18.627 INFO:teuthology.orchestra.run.smithi052.stdout:config.status: creating include/config.h 2024-05-04T12:20:18.639 INFO:teuthology.orchestra.run.smithi052.stdout:config.status: executing libtool commands 2024-05-04T12:20:18.686 INFO:teuthology.orchestra.run.smithi052.stdout:Building include 2024-05-04T12:20:18.690 INFO:teuthology.orchestra.run.smithi052.stdout:Building lib 2024-05-04T12:20:18.694 INFO:teuthology.orchestra.run.smithi052.stdout: [LTDEP] 2024-05-04T12:20:18.850 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] dataascii.lo 2024-05-04T12:20:18.986 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] databin.lo 2024-05-04T12:20:19.168 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] datapid.lo 2024-05-04T12:20:19.291 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] file_lock.lo 2024-05-04T12:20:19.471 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] forker.lo 2024-05-04T12:20:19.620 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] pattern.lo 2024-05-04T12:20:19.748 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] open_flags.lo 2024-05-04T12:20:19.895 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] random_range.lo 2024-05-04T12:20:20.151 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] string_to_tokens.lo 2024-05-04T12:20:20.248 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] str_to_bytes.lo 2024-05-04T12:20:20.387 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] tlibio.lo 2024-05-04T12:20:20.662 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] write_log.lo 2024-05-04T12:20:20.724 INFO:teuthology.orchestra.run.smithi052.stderr:write_log.c: In function 'wlog_record_write': 2024-05-04T12:20:20.724 INFO:teuthology.orchestra.run.smithi052.stderr:write_log.c:205:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:20.724 INFO:teuthology.orchestra.run.smithi052.stderr: 205 | write(wfile->w_afd, wbuf, reclen); 2024-05-04T12:20:20.724 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:20.725 INFO:teuthology.orchestra.run.smithi052.stderr:write_log.c:209:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:20.725 INFO:teuthology.orchestra.run.smithi052.stderr: 209 | write(wfile->w_rfd, wbuf, reclen); 2024-05-04T12:20:20.725 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:20.931 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] random.lo 2024-05-04T12:20:21.034 INFO:teuthology.orchestra.run.smithi052.stdout: [LD] 2024-05-04T12:20:21.109 INFO:teuthology.orchestra.run.smithi052.stdout:Building ltp 2024-05-04T12:20:21.112 INFO:teuthology.orchestra.run.smithi052.stdout: [DEP] 2024-05-04T12:20:21.272 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] doio 2024-05-04T12:20:22.008 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fsstress 2024-05-04T12:20:23.563 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fsx 2024-05-04T12:20:23.591 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:20:23.593 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:20:24.388 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] growfiles 2024-05-04T12:20:24.952 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] iogen 2024-05-04T12:20:25.081 INFO:teuthology.orchestra.run.smithi052.stderr:iogen.c: In function 'main': 2024-05-04T12:20:25.082 INFO:teuthology.orchestra.run.smithi052.stderr:iogen.c:280:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:25.082 INFO:teuthology.orchestra.run.smithi052.stderr: 280 | write(outfd, (char *)&req, sizeof(req)); 2024-05-04T12:20:25.082 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:25.084 INFO:teuthology.orchestra.run.smithi052.stderr:iogen.c: In function 'parse_cmdline': 2024-05-04T12:20:25.084 INFO:teuthology.orchestra.run.smithi052.stderr:iogen.c:1535:17: warning: ignoring return value of 'getcwd' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:25.084 INFO:teuthology.orchestra.run.smithi052.stderr: 1535 | getcwd(fptr->f_path, 2024-05-04T12:20:25.084 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:25.085 INFO:teuthology.orchestra.run.smithi052.stderr: 1536 | sizeof(fptr->f_path)-1); 2024-05-04T12:20:25.085 INFO:teuthology.orchestra.run.smithi052.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:25.380 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-stress 2024-05-04T12:20:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:aio-stress.c: In function 'setup_shared_mem': 2024-05-04T12:20:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:aio-stress.c:1013:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:25.505 INFO:teuthology.orchestra.run.smithi052.stderr: 1013 | ftruncate(fd, total_ram); 2024-05-04T12:20:25.505 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:25.851 INFO:teuthology.orchestra.run.smithi052.stdout:Building src 2024-05-04T12:20:25.861 INFO:teuthology.orchestra.run.smithi052.stdout: [DEP] 2024-05-04T12:20:28.092 INFO:teuthology.orchestra.run.smithi052.stdout: [DEP] 2024-05-04T12:20:28.128 INFO:teuthology.orchestra.run.smithi052.stdout: [DEP] 2024-05-04T12:20:28.136 INFO:teuthology.orchestra.run.smithi052.stdout: [DEP] 2024-05-04T12:20:28.409 INFO:teuthology.orchestra.run.smithi052.stdout: [DEP] 2024-05-04T12:20:28.436 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] dirstress 2024-05-04T12:20:28.723 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fill 2024-05-04T12:20:28.880 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fill2 2024-05-04T12:20:29.110 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] getpagesize 2024-05-04T12:20:29.240 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] holes 2024-05-04T12:20:29.435 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] lstat64 2024-05-04T12:20:29.625 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] nametest 2024-05-04T12:20:29.759 INFO:teuthology.orchestra.run.smithi052.stderr:nametest.c: In function 'main': 2024-05-04T12:20:29.759 INFO:teuthology.orchestra.run.smithi052.stderr:nametest.c:236:33: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:29.759 INFO:teuthology.orchestra.run.smithi052.stderr: 236 | write(1, "\n", 1); 2024-05-04T12:20:29.759 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~ 2024-05-04T12:20:29.759 INFO:teuthology.orchestra.run.smithi052.stderr:nametest.c:238:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:29.759 INFO:teuthology.orchestra.run.smithi052.stderr: 238 | write(1, ".", 1); 2024-05-04T12:20:29.759 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~ 2024-05-04T12:20:29.759 INFO:teuthology.orchestra.run.smithi052.stderr:nametest.c:288:33: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:29.760 INFO:teuthology.orchestra.run.smithi052.stderr: 288 | write(1, ".", 1); 2024-05-04T12:20:29.760 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~ 2024-05-04T12:20:29.889 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] permname 2024-05-04T12:20:30.093 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] randholes 2024-05-04T12:20:30.367 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] runas 2024-05-04T12:20:30.543 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] truncfile 2024-05-04T12:20:30.731 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] usemem 2024-05-04T12:20:30.887 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] mmapcat 2024-05-04T12:20:30.995 INFO:teuthology.orchestra.run.smithi052.stderr:mmapcat.c: In function 'main': 2024-05-04T12:20:30.995 INFO:teuthology.orchestra.run.smithi052.stderr:mmapcat.c:29:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:30.995 INFO:teuthology.orchestra.run.smithi052.stderr: 29 | write(1,ptr2,ptr - ptr2); 2024-05-04T12:20:30.996 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:31.032 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] append_reader 2024-05-04T12:20:31.185 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] append_writer 2024-05-04T12:20:31.336 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] dirperf 2024-05-04T12:20:31.588 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] metaperf 2024-05-04T12:20:31.717 INFO:teuthology.orchestra.run.smithi052.stderr:metaperf.c: In function 'crfiles': 2024-05-04T12:20:31.718 INFO:teuthology.orchestra.run.smithi052.stderr:metaperf.c:166:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:31.718 INFO:teuthology.orchestra.run.smithi052.stderr: 166 | write(fd, buf, fsize); 2024-05-04T12:20:31.718 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:31.719 INFO:teuthology.orchestra.run.smithi052.stderr:metaperf.c: In function 't_linkun': 2024-05-04T12:20:31.719 INFO:teuthology.orchestra.run.smithi052.stderr:metaperf.c:454:25: warning: ignoring return value of 'link' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:31.719 INFO:teuthology.orchestra.run.smithi052.stderr: 454 | link("a", *fnp); 2024-05-04T12:20:31.719 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~ 2024-05-04T12:20:31.719 INFO:teuthology.orchestra.run.smithi052.stderr:metaperf.c: In function 't_chown': 2024-05-04T12:20:31.719 INFO:teuthology.orchestra.run.smithi052.stderr:metaperf.c:404:33: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:31.719 INFO:teuthology.orchestra.run.smithi052.stderr: 404 | chown(*fnp, 2, -1); 2024-05-04T12:20:31.719 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:31.719 INFO:teuthology.orchestra.run.smithi052.stderr:metaperf.c:406:33: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:31.720 INFO:teuthology.orchestra.run.smithi052.stderr: 406 | chown(*fnp, 1, -1); 2024-05-04T12:20:31.720 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:31.720 INFO:teuthology.orchestra.run.smithi052.stderr:metaperf.c: In function 'i_chown': 2024-05-04T12:20:31.720 INFO:teuthology.orchestra.run.smithi052.stderr:metaperf.c:291:17: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:31.720 INFO:teuthology.orchestra.run.smithi052.stderr: 291 | chown(*fnp, 1, -1); 2024-05-04T12:20:31.720 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:31.720 INFO:teuthology.orchestra.run.smithi052.stderr:metaperf.c: In function 'main': 2024-05-04T12:20:31.720 INFO:teuthology.orchestra.run.smithi052.stderr:metaperf.c:152:9: warning: ignoring return value of 'chdir' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:31.720 INFO:teuthology.orchestra.run.smithi052.stderr: 152 | chdir(".."); 2024-05-04T12:20:31.720 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~ 2024-05-04T12:20:31.897 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] devzero 2024-05-04T12:20:32.078 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] feature 2024-05-04T12:20:32.317 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] alloc 2024-05-04T12:20:32.447 INFO:teuthology.orchestra.run.smithi052.stderr:alloc.c: In function 'main': 2024-05-04T12:20:32.447 INFO:teuthology.orchestra.run.smithi052.stderr:alloc.c:247:17: warning: ignoring return value of 'mkstemp' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:32.447 INFO:teuthology.orchestra.run.smithi052.stderr: 247 | mkstemp(tmpfile); 2024-05-04T12:20:32.448 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~ 2024-05-04T12:20:32.548 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fault 2024-05-04T12:20:32.714 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fstest 2024-05-04T12:20:32.974 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_access_root 2024-05-04T12:20:33.081 INFO:teuthology.orchestra.run.smithi052.stderr:t_access_root.c: In function 'main': 2024-05-04T12:20:33.081 INFO:teuthology.orchestra.run.smithi052.stderr:t_access_root.c:55:5: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:33.081 INFO:teuthology.orchestra.run.smithi052.stderr: 55 | system(cmd); 2024-05-04T12:20:33.081 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~ 2024-05-04T12:20:33.121 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] godown 2024-05-04T12:20:33.286 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] resvtest 2024-05-04T12:20:33.480 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] writemod 2024-05-04T12:20:33.619 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] writev_on_pagefault 2024-05-04T12:20:33.766 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] makeextents 2024-05-04T12:20:33.933 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] itrash 2024-05-04T12:20:34.102 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] rename 2024-05-04T12:20:34.240 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] multi_open_unlink 2024-05-04T12:20:34.400 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] unwritten_sync 2024-05-04T12:20:34.600 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] genhashnames 2024-05-04T12:20:34.800 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_holes 2024-05-04T12:20:34.965 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_writev 2024-05-04T12:20:35.125 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_truncate_cmtime 2024-05-04T12:20:35.283 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] dirhash_collide 2024-05-04T12:20:35.467 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_rename_overwrite 2024-05-04T12:20:35.597 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] holetest 2024-05-04T12:20:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:holetest.c: In function 'pt_write_marker': 2024-05-04T12:20:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:holetest.c:140:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:35.711 INFO:teuthology.orchestra.run.smithi052.stderr: 140 | pwrite(fd, &tid, sizeof(tid), i * page_size + pgoff); 2024-05-04T12:20:35.711 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:35.832 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_truncate_self 2024-05-04T12:20:35.970 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] af_unix 2024-05-04T12:20:36.120 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_stale_pmd 2024-05-04T12:20:36.223 INFO:teuthology.orchestra.run.smithi052.stderr:t_mmap_stale_pmd.c: In function 'main': 2024-05-04T12:20:36.223 INFO:teuthology.orchestra.run.smithi052.stderr:t_mmap_stale_pmd.c:42:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:36.223 INFO:teuthology.orchestra.run.smithi052.stderr: 42 | ftruncate(fd, 0); 2024-05-04T12:20:36.223 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~ 2024-05-04T12:20:36.223 INFO:teuthology.orchestra.run.smithi052.stderr:t_mmap_stale_pmd.c:43:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:36.223 INFO:teuthology.orchestra.run.smithi052.stderr: 43 | ftruncate(fd, MiB(4)); 2024-05-04T12:20:36.223 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:36.223 INFO:teuthology.orchestra.run.smithi052.stderr:t_mmap_stale_pmd.c:55:9: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:36.223 INFO:teuthology.orchestra.run.smithi052.stderr: 55 | pwrite(fd, buffer, strlen(buffer), MiB(2)); 2024-05-04T12:20:36.224 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:36.259 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_cow_race 2024-05-04T12:20:36.377 INFO:teuthology.orchestra.run.smithi052.stderr:t_mmap_cow_race.c: In function 'main': 2024-05-04T12:20:36.377 INFO:teuthology.orchestra.run.smithi052.stderr:t_mmap_cow_race.c:61:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:36.377 INFO:teuthology.orchestra.run.smithi052.stderr: 61 | ftruncate(fd, 0); 2024-05-04T12:20:36.377 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~ 2024-05-04T12:20:36.377 INFO:teuthology.orchestra.run.smithi052.stderr:t_mmap_cow_race.c:62:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:36.377 INFO:teuthology.orchestra.run.smithi052.stderr: 62 | ftruncate(fd, MiB(2)); 2024-05-04T12:20:36.377 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:36.428 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_fallocate 2024-05-04T12:20:36.573 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fsync-err 2024-05-04T12:20:36.762 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_write_ro 2024-05-04T12:20:36.921 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_ext4_dax_journal_corruption 2024-05-04T12:20:37.030 INFO:teuthology.orchestra.run.smithi052.stderr:t_ext4_dax_journal_corruption.c: In function 'main': 2024-05-04T12:20:37.030 INFO:teuthology.orchestra.run.smithi052.stderr:t_ext4_dax_journal_corruption.c:61:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:37.030 INFO:teuthology.orchestra.run.smithi052.stderr: 61 | ftruncate(fd, 0); 2024-05-04T12:20:37.031 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~ 2024-05-04T12:20:37.079 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_ext4_dax_inline_corruption 2024-05-04T12:20:37.230 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_ofd_locks 2024-05-04T12:20:37.427 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_collision 2024-05-04T12:20:37.655 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] mmap-write-concurrent 2024-05-04T12:20:37.832 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_get_file_time 2024-05-04T12:20:37.975 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_create_short_dirs 2024-05-04T12:20:38.128 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_create_long_dirs 2024-05-04T12:20:38.287 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_enospc 2024-05-04T12:20:38.537 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_writev_overlap 2024-05-04T12:20:38.720 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] checkpoint_journal 2024-05-04T12:20:38.878 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] mmap-rw-fault 2024-05-04T12:20:39.067 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] allocstale 2024-05-04T12:20:39.242 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_cow_memory_failure 2024-05-04T12:20:39.438 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fake-dump-rootino 2024-05-04T12:20:39.586 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] dio-buf-fault 2024-05-04T12:20:39.734 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] rewinddir-test 2024-05-04T12:20:39.905 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] readdir-while-renames 2024-05-04T12:20:40.064 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] xfsctl 2024-05-04T12:20:40.257 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] bstat 2024-05-04T12:20:40.484 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mtab 2024-05-04T12:20:40.596 INFO:teuthology.orchestra.run.smithi052.stderr:t_mtab.c: In function 'update_mtab': 2024-05-04T12:20:40.596 INFO:teuthology.orchestra.run.smithi052.stderr:t_mtab.c:256:17: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:40.596 INFO:teuthology.orchestra.run.smithi052.stderr: 256 | chown (mounted_temp, sbuf.st_uid, sbuf.st_gid); 2024-05-04T12:20:40.596 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:40.676 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] getdevicesize 2024-05-04T12:20:40.812 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] preallo_rw_pattern_reader 2024-05-04T12:20:40.931 INFO:teuthology.orchestra.run.smithi052.stderr:In file included from preallo_rw_pattern_reader.c:2: 2024-05-04T12:20:40.931 INFO:teuthology.orchestra.run.smithi052.stderr:iopat.c: In function 'main': 2024-05-04T12:20:40.931 INFO:teuthology.orchestra.run.smithi052.stderr:iopat.c:38:4: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:40.931 INFO:teuthology.orchestra.run.smithi052.stderr: 38 | read(fd, &x, 1048576); 2024-05-04T12:20:40.931 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:40.967 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] preallo_rw_pattern_writer 2024-05-04T12:20:41.094 INFO:teuthology.orchestra.run.smithi052.stderr:In file included from preallo_rw_pattern_writer.c:2: 2024-05-04T12:20:41.094 INFO:teuthology.orchestra.run.smithi052.stderr:iopat.c: In function 'main': 2024-05-04T12:20:41.094 INFO:teuthology.orchestra.run.smithi052.stderr:iopat.c:34:4: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:41.094 INFO:teuthology.orchestra.run.smithi052.stderr: 34 | write(fd, &x, 1048576); 2024-05-04T12:20:41.094 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:41.134 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] ftrunc 2024-05-04T12:20:41.282 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] trunc 2024-05-04T12:20:41.464 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fs_perms 2024-05-04T12:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr:fs_perms.c: In function 'main': 2024-05-04T12:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr:fs_perms.c:54:4: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr: 54 | system("rm -f test.file"); 2024-05-04T12:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr:fs_perms.c: In function 'testsetup': 2024-05-04T12:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr:fs_perms.c:67:4: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr: 67 | system("cp testx.file test.file"); 2024-05-04T12:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr:fs_perms.c:69:4: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr: 69 | chown("test.file",cuserId,cgroupId); 2024-05-04T12:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:fs_perms.c: In function 'testfperm': 2024-05-04T12:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:fs_perms.c:123:5: warning: ignoring return value of 'seteuid' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr: 123 | seteuid(0); 2024-05-04T12:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~ 2024-05-04T12:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:fs_perms.c:124:5: warning: ignoring return value of 'setegid' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr: 124 | setegid(0); 2024-05-04T12:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~ 2024-05-04T12:20:41.634 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] testx 2024-05-04T12:20:41.751 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] looptest 2024-05-04T12:20:41.943 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] locktest 2024-05-04T12:20:42.343 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] unwritten_mmap 2024-05-04T12:20:42.524 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] bulkstat_unlink_test 2024-05-04T12:20:42.658 INFO:teuthology.orchestra.run.smithi052.stderr:bulkstat_unlink_test.c: In function 'main': 2024-05-04T12:20:42.658 INFO:teuthology.orchestra.run.smithi052.stderr:bulkstat_unlink_test.c:106:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:42.658 INFO:teuthology.orchestra.run.smithi052.stderr: 106 | write(fd[i], fname, sizeof(fname)); 2024-05-04T12:20:42.659 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:42.659 INFO:teuthology.orchestra.run.smithi052.stderr:bulkstat_unlink_test.c:178:17: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:42.659 INFO:teuthology.orchestra.run.smithi052.stderr: 178 | system(fname); 2024-05-04T12:20:42.659 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~ 2024-05-04T12:20:42.750 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] deduperace 2024-05-04T12:20:42.956 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] bulkstat_unlink_test_modified 2024-05-04T12:20:43.089 INFO:teuthology.orchestra.run.smithi052.stderr:bulkstat_unlink_test_modified.c: In function 'main': 2024-05-04T12:20:43.090 INFO:teuthology.orchestra.run.smithi052.stderr:bulkstat_unlink_test_modified.c:83:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.090 INFO:teuthology.orchestra.run.smithi052.stderr: 83 | write(fd, fname, sizeof(fname)); 2024-05-04T12:20:43.090 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.090 INFO:teuthology.orchestra.run.smithi052.stderr:bulkstat_unlink_test_modified.c:139:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.090 INFO:teuthology.orchestra.run.smithi052.stderr: 139 | write(fd, fname, sizeof(fname)); 2024-05-04T12:20:43.090 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.090 INFO:teuthology.orchestra.run.smithi052.stderr:bulkstat_unlink_test_modified.c:185:9: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.090 INFO:teuthology.orchestra.run.smithi052.stderr: 185 | system(fname); 2024-05-04T12:20:43.090 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~ 2024-05-04T12:20:43.175 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_dir_offset 2024-05-04T12:20:43.326 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_futimens 2024-05-04T12:20:43.465 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_immutable 2024-05-04T12:20:43.612 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c: In function 'create_test_area': 2024-05-04T12:20:43.613 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:2061:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.613 INFO:teuthology.orchestra.run.smithi052.stderr: 2061 | asprintf(&path, "%s/immutable.d", dir); 2024-05-04T12:20:43.613 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.613 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:2096:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.613 INFO:teuthology.orchestra.run.smithi052.stderr: 2096 | asprintf(&path, "%s/append-only.d", dir); 2024-05-04T12:20:43.613 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.614 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c: In function 'remove_test_area': 2024-05-04T12:20:43.614 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:2152:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.614 INFO:teuthology.orchestra.run.smithi052.stderr: 2152 | asprintf(&path, "%s/immutable.d", dir); 2024-05-04T12:20:43.614 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.614 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:2163:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.614 INFO:teuthology.orchestra.run.smithi052.stderr: 2163 | asprintf(&path, "%s/empty-immutable.d", dir); 2024-05-04T12:20:43.614 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.615 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:2175:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.615 INFO:teuthology.orchestra.run.smithi052.stderr: 2175 | asprintf(&path, "%s/append-only.d", dir); 2024-05-04T12:20:43.615 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.615 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:2186:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.615 INFO:teuthology.orchestra.run.smithi052.stderr: 2186 | asprintf(&path, "%s/empty-append-only.d", dir); 2024-05-04T12:20:43.615 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.615 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:2197:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.615 INFO:teuthology.orchestra.run.smithi052.stderr: 2197 | asprintf(&path, "%s/append-only.f", dir); 2024-05-04T12:20:43.615 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.616 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:2209:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.616 INFO:teuthology.orchestra.run.smithi052.stderr: 2209 | asprintf(&path, "%s/immutable.f", dir); 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c: In function 'check_test_area': 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1788:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr: 1788 | asprintf(&path, "%s/", dir); 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1800:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr: 1800 | asprintf(&path, "%s/immutable.f", dir); 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1812:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr: 1812 | asprintf(&path, "%s/append-only.f", dir); 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1824:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr: 1824 | asprintf(&path, "%s/immutable.d", dir); 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.617 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1837:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.618 INFO:teuthology.orchestra.run.smithi052.stderr: 1837 | asprintf(&path, "%s/append-only.d", dir); 2024-05-04T12:20:43.618 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.618 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1850:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.618 INFO:teuthology.orchestra.run.smithi052.stderr: 1850 | asprintf(&path, "%s/immutable.d/file", dir); 2024-05-04T12:20:43.618 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.618 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1862:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.618 INFO:teuthology.orchestra.run.smithi052.stderr: 1862 | asprintf(&path, "%s/append-only.d/file", dir); 2024-05-04T12:20:43.618 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.618 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1874:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.618 INFO:teuthology.orchestra.run.smithi052.stderr: 1874 | asprintf(&path, "%s/immutable.d/dir", dir); 2024-05-04T12:20:43.618 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.618 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1887:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.618 INFO:teuthology.orchestra.run.smithi052.stderr: 1887 | asprintf(&path, "%s/append-only.d/dir", dir); 2024-05-04T12:20:43.618 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.620 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c: In function 'test_immutable': 2024-05-04T12:20:43.620 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:151:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.621 INFO:teuthology.orchestra.run.smithi052.stderr: 151 | asprintf(&path, "%s/immutable.f", dir); 2024-05-04T12:20:43.621 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.621 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:381:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.621 INFO:teuthology.orchestra.run.smithi052.stderr: 381 | asprintf(&linkpath, "%s/immutable.f.hardlink", dir); 2024-05-04T12:20:43.621 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.621 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:408:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.621 INFO:teuthology.orchestra.run.smithi052.stderr: 408 | chown(path, 0, 0); 2024-05-04T12:20:43.621 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.621 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:485:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.621 INFO:teuthology.orchestra.run.smithi052.stderr: 485 | asprintf(&linkpath, "%s/immutable.f.newname", dir); 2024-05-04T12:20:43.621 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.621 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:532:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.621 INFO:teuthology.orchestra.run.smithi052.stderr: 532 | asprintf(&path, "%s/immutable.d/file", dir); 2024-05-04T12:20:43.621 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:553:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr: 553 | chown(path, 0, 0); 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:556:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr: 556 | asprintf(&linkpath, "%s/immutable.d/file.link", dir); 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:575:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr: 575 | asprintf(&linkpath, "%s/immutable.d/file.newname", dir); 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:595:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr: 595 | asprintf(&path, "%s/immutable.d/newfile", dir); 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:619:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr: 619 | asprintf(&path, "%s/immutable.d/newdir", dir); 2024-05-04T12:20:43.622 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.623 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:631:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.623 INFO:teuthology.orchestra.run.smithi052.stderr: 631 | asprintf(&path, "%s/immutable.d/dir/newfile-%d", dir, getuid()); 2024-05-04T12:20:43.623 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.623 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:653:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.623 INFO:teuthology.orchestra.run.smithi052.stderr: 653 | chown(path, 0, 0); 2024-05-04T12:20:43.623 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.623 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:657:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.623 INFO:teuthology.orchestra.run.smithi052.stderr: 657 | asprintf(&path, "%s/immutable.d/dir", dir); 2024-05-04T12:20:43.623 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.623 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:668:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.623 INFO:teuthology.orchestra.run.smithi052.stderr: 668 | asprintf(&path, "%s/immutable.d", dir); 2024-05-04T12:20:43.623 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.623 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:705:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.623 INFO:teuthology.orchestra.run.smithi052.stderr: 705 | chown(path, 0, 0); 2024-05-04T12:20:43.624 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.624 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:782:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.624 INFO:teuthology.orchestra.run.smithi052.stderr: 782 | asprintf(&path, "%s/empty-immutable.d", dir); 2024-05-04T12:20:43.624 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.626 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c: In function 'test_append': 2024-05-04T12:20:43.626 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:822:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.626 INFO:teuthology.orchestra.run.smithi052.stderr: 822 | asprintf(&path, "%s/append-only.f", dir); 2024-05-04T12:20:43.626 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1024:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr: 1024 | asprintf(&linkpath, "%s/append-only.f.hardlink", dir); 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1051:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr: 1051 | chown(path, 0, 0); 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1126:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr: 1126 | asprintf(&linkpath, "%s/append-only.f.newname", dir); 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1495:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr: 1495 | asprintf(&path, "%s/append-only.d/file", dir); 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1516:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr: 1516 | chown(path, 0, 0); 2024-05-04T12:20:43.627 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.628 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1519:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.628 INFO:teuthology.orchestra.run.smithi052.stderr: 1519 | asprintf(&linkpath, "%s/append-only.d/file.link-%d", dir, getuid()); 2024-05-04T12:20:43.628 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.628 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1529:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.628 INFO:teuthology.orchestra.run.smithi052.stderr: 1529 | asprintf(&linkpath, "%s/append-only.d/file.symlink-%d", dir, getuid()); 2024-05-04T12:20:43.628 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.628 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1539:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.628 INFO:teuthology.orchestra.run.smithi052.stderr: 1539 | asprintf(&linkpath, "%s/append-only.d/file.newname", dir); 2024-05-04T12:20:43.628 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.628 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1559:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.628 INFO:teuthology.orchestra.run.smithi052.stderr: 1559 | asprintf(&path, "%s/append-only.d/newfile-%d", dir, getuid()); 2024-05-04T12:20:43.628 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.628 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1573:11: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.628 INFO:teuthology.orchestra.run.smithi052.stderr: 1573 | asprintf(&path, "%s/append-only.d/newdev-%d", dir, getuid()); 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1586:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr: 1586 | asprintf(&path, "%s/append-only.d/newdir-%d", dir, getuid()); 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1596:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr: 1596 | asprintf(&path, "%s/append-only.d/newdir-%d/newfile-%d", dir, getuid(), getuid()); 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1618:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr: 1618 | chown(path, 0, 0); 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1622:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr: 1622 | asprintf(&path, "%s/append-only.d/dir/newfile-%d", dir, getuid()); 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1644:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.629 INFO:teuthology.orchestra.run.smithi052.stderr: 1644 | chown(path, 0, 0); 2024-05-04T12:20:43.630 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.630 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1648:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.630 INFO:teuthology.orchestra.run.smithi052.stderr: 1648 | asprintf(&path, "%s/append-only.d/dir", dir); 2024-05-04T12:20:43.630 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.630 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1659:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.630 INFO:teuthology.orchestra.run.smithi052.stderr: 1659 | asprintf(&path, "%s/append-only.d", dir); 2024-05-04T12:20:43.630 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.630 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1692:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.630 INFO:teuthology.orchestra.run.smithi052.stderr: 1692 | chown(path, 0, 0); 2024-05-04T12:20:43.630 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~ 2024-05-04T12:20:43.630 INFO:teuthology.orchestra.run.smithi052.stderr:t_immutable.c:1769:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:43.630 INFO:teuthology.orchestra.run.smithi052.stderr: 1769 | asprintf(&path, "%s/empty-append-only.d", dir); 2024-05-04T12:20:43.630 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:44.210 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] stale_handle 2024-05-04T12:20:44.326 INFO:teuthology.orchestra.run.smithi052.stderr:stale_handle.c: In function 'main': 2024-05-04T12:20:44.326 INFO:teuthology.orchestra.run.smithi052.stderr:stale_handle.c:102:9: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:44.326 INFO:teuthology.orchestra.run.smithi052.stderr: 102 | system("echo 3 > /proc/sys/vm/drop_caches"); 2024-05-04T12:20:44.326 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:44.377 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] pwrite_mmap_blocked 2024-05-04T12:20:44.520 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_dir_offset2 2024-05-04T12:20:44.707 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] seek_sanity_test 2024-05-04T12:20:44.851 INFO:teuthology.orchestra.run.smithi052.stderr:seek_sanity_test.c: In function 'test18': 2024-05-04T12:20:44.851 INFO:teuthology.orchestra.run.smithi052.stderr:seek_sanity_test.c:460:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:44.851 INFO:teuthology.orchestra.run.smithi052.stderr: 460 | ftruncate(fd, 0); 2024-05-04T12:20:44.851 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~ 2024-05-04T12:20:44.853 INFO:teuthology.orchestra.run.smithi052.stderr:seek_sanity_test.c: In function 'test_basic_support': 2024-05-04T12:20:44.853 INFO:teuthology.orchestra.run.smithi052.stderr:seek_sanity_test.c:1207:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:44.853 INFO:teuthology.orchestra.run.smithi052.stderr: 1207 | ftruncate(fd, 0); 2024-05-04T12:20:44.854 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~ 2024-05-04T12:20:44.854 INFO:teuthology.orchestra.run.smithi052.stderr:seek_sanity_test.c:1256:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:44.854 INFO:teuthology.orchestra.run.smithi052.stderr: 1256 | ftruncate(fd, 0); 2024-05-04T12:20:44.854 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~ 2024-05-04T12:20:44.854 INFO:teuthology.orchestra.run.smithi052.stderr:seek_sanity_test.c: In function 'get_io_sizes': 2024-05-04T12:20:44.854 INFO:teuthology.orchestra.run.smithi052.stderr:seek_sanity_test.c:88:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:44.854 INFO:teuthology.orchestra.run.smithi052.stderr: 88 | ftruncate(fd, 0); 2024-05-04T12:20:44.854 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~ 2024-05-04T12:20:44.854 INFO:teuthology.orchestra.run.smithi052.stderr:seek_sanity_test.c:89:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:44.855 INFO:teuthology.orchestra.run.smithi052.stderr: 89 | pwrite(fd, "a", 1, offset); 2024-05-04T12:20:44.855 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:44.855 INFO:teuthology.orchestra.run.smithi052.stderr:seek_sanity_test.c:98:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:44.855 INFO:teuthology.orchestra.run.smithi052.stderr: 98 | ftruncate(fd, 0); 2024-05-04T12:20:44.855 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~ 2024-05-04T12:20:44.855 INFO:teuthology.orchestra.run.smithi052.stderr:seek_sanity_test.c:99:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:44.855 INFO:teuthology.orchestra.run.smithi052.stderr: 99 | pwrite(fd, "a", 1, offset); 2024-05-04T12:20:44.855 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:45.199 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] seek_copy_test 2024-05-04T12:20:45.397 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_readdir_1 2024-05-04T12:20:45.519 INFO:teuthology.orchestra.run.smithi052.stderr:t_readdir_1.c: In function 'main': 2024-05-04T12:20:45.519 INFO:teuthology.orchestra.run.smithi052.stderr:t_readdir_1.c:30:25: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:45.519 INFO:teuthology.orchestra.run.smithi052.stderr: 30 | read(fd, buf, 100); 2024-05-04T12:20:45.519 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:45.556 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_readdir_2 2024-05-04T12:20:45.700 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fsync-tester 2024-05-04T12:20:45.820 INFO:teuthology.orchestra.run.smithi052.stderr:fsync-tester.c: In function 'main': 2024-05-04T12:20:45.820 INFO:teuthology.orchestra.run.smithi052.stderr:fsync-tester.c:391:17: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:45.820 INFO:teuthology.orchestra.run.smithi052.stderr: 391 | read(fd, &seed, sizeof(seed)); 2024-05-04T12:20:45.821 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:45.821 INFO:teuthology.orchestra.run.smithi052.stderr:fsync-tester.c:541:17: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:45.821 INFO:teuthology.orchestra.run.smithi052.stderr: 541 | system("reboot -fn"); 2024-05-04T12:20:45.821 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:45.821 INFO:teuthology.orchestra.run.smithi052.stderr:fsync-tester.c: In function 'drop_all_caches': 2024-05-04T12:20:45.821 INFO:teuthology.orchestra.run.smithi052.stderr:fsync-tester.c:90:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:45.821 INFO:teuthology.orchestra.run.smithi052.stderr: 90 | write(fd, value, sizeof(value)-1); 2024-05-04T12:20:45.821 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:45.984 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] nsexec 2024-05-04T12:20:46.173 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] cloner 2024-05-04T12:20:46.368 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] renameat2 2024-05-04T12:20:46.544 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_getcwd 2024-05-04T12:20:46.663 INFO:teuthology.orchestra.run.smithi052.stderr:t_getcwd.c: In function 'test_getcwd': 2024-05-04T12:20:46.663 INFO:teuthology.orchestra.run.smithi052.stderr:t_getcwd.c:21:17: warning: ignoring return value of 'getcwd' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:46.663 INFO:teuthology.orchestra.run.smithi052.stderr: 21 | getcwd(cur_cwd, BUF_SIZE); 2024-05-04T12:20:46.663 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:46.715 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] e4compact 2024-05-04T12:20:46.937 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] test-nextquota 2024-05-04T12:20:47.111 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] punch-alternating 2024-05-04T12:20:47.278 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] attr-list-by-handle-cursor-test 2024-05-04T12:20:47.445 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] listxattr 2024-05-04T12:20:47.588 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] dio-interleaved 2024-05-04T12:20:47.745 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_dir_type 2024-05-04T12:20:47.896 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] dio-invalidate-cache 2024-05-04T12:20:48.015 INFO:teuthology.orchestra.run.smithi052.stderr:dio-invalidate-cache.c: In function 'main': 2024-05-04T12:20:48.016 INFO:teuthology.orchestra.run.smithi052.stderr:dio-invalidate-cache.c:286:17: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:48.016 INFO:teuthology.orchestra.run.smithi052.stderr: 286 | write(fd, buf, blksz * nr_child); 2024-05-04T12:20:48.016 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:48.120 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] stat_test 2024-05-04T12:20:48.429 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_encrypted_d_revalidate 2024-05-04T12:20:48.602 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] attr_replace_test 2024-05-04T12:20:48.762 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] swapon 2024-05-04T12:20:48.939 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] mkswap 2024-05-04T12:20:49.099 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_attr_corruption 2024-05-04T12:20:49.247 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_open_tmpfiles 2024-05-04T12:20:49.429 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fscrypt-crypt-util 2024-05-04T12:20:50.133 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] bulkstat_null_ocount 2024-05-04T12:20:50.279 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] splice-test 2024-05-04T12:20:50.452 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] chprojid_fail 2024-05-04T12:20:50.600 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] detached_mounts_propagation 2024-05-04T12:20:50.799 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] ext4_resize 2024-05-04T12:20:50.956 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_readdir_3 2024-05-04T12:20:51.160 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] splice2pipe 2024-05-04T12:20:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:splice2pipe.c: In function 'prepare_pipe': 2024-05-04T12:20:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:splice2pipe.c:57:17: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:51.287 INFO:teuthology.orchestra.run.smithi052.stderr: 57 | write(p[1], buffer, n); 2024-05-04T12:20:51.287 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:splice2pipe.c:65:17: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:51.287 INFO:teuthology.orchestra.run.smithi052.stderr: 65 | read(p[0], buffer, n); 2024-05-04T12:20:51.287 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:51.350 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] uuid_ioctl 2024-05-04T12:20:51.520 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_snapshot_deleted_subvolume 2024-05-04T12:20:51.695 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fiemap-fault 2024-05-04T12:20:51.857 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fiemap-tester 2024-05-04T12:20:52.131 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_stripealign 2024-05-04T12:20:52.287 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] open_by_handle 2024-05-04T12:20:52.546 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_dio 2024-05-04T12:20:52.716 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] uring_read_fault 2024-05-04T12:20:52.913 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] xfsfind 2024-05-04T12:20:53.119 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_reflink_read_race 2024-05-04T12:20:53.331 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fssum 2024-05-04T12:20:53.638 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:20:53.641 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:20:53.823 INFO:teuthology.orchestra.run.smithi052.stdout:Building log-writes 2024-05-04T12:20:53.830 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] replay-log 2024-05-04T12:20:54.192 INFO:teuthology.orchestra.run.smithi052.stdout:Building perf 2024-05-04T12:20:54.197 INFO:teuthology.orchestra.run.smithi052.stdout:Building aio-dio-regress 2024-05-04T12:20:54.204 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aiocp 2024-05-04T12:20:54.455 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-append-write-fallocate-race 2024-05-04T12:20:54.648 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-append-write-read-race 2024-05-04T12:20:54.847 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-cow-race 2024-05-04T12:20:55.003 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-cycle-write 2024-05-04T12:20:55.115 INFO:teuthology.orchestra.run.smithi052.stderr:aio-dio-cycle-write.c: In function 'init_test': 2024-05-04T12:20:55.115 INFO:teuthology.orchestra.run.smithi052.stderr:aio-dio-cycle-write.c:83:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:55.115 INFO:teuthology.orchestra.run.smithi052.stderr: 83 | ftruncate(fd, buf_size); 2024-05-04T12:20:55.115 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:55.210 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-eof-race 2024-05-04T12:20:55.415 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-extend-stat 2024-05-04T12:20:55.580 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-fcntl-race 2024-05-04T12:20:55.755 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-hole-filling-race 2024-05-04T12:20:55.912 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-invalidate-failure 2024-05-04T12:20:56.079 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-invalidate-readahead 2024-05-04T12:20:56.246 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aiodio_sparse2 2024-05-04T12:20:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:aiodio_sparse2.c: In function 'aiodio_sparse': 2024-05-04T12:20:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:aiodio_sparse2.c:111:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:56.349 INFO:teuthology.orchestra.run.smithi052.stderr: 111 | ftruncate(fd, filesize); 2024-05-04T12:20:56.349 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:aiodio_sparse2.c: In function 'dirty_freeblocks': 2024-05-04T12:20:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:aiodio_sparse2.c:274:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-05-04T12:20:56.349 INFO:teuthology.orchestra.run.smithi052.stderr: 274 | ftruncate(fd, size); 2024-05-04T12:20:56.349 INFO:teuthology.orchestra.run.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-04T12:20:56.476 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-subblock-eof-read 2024-05-04T12:20:56.627 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-write-verify 2024-05-04T12:20:56.833 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-05-04T12:20:56.968 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-05-04T12:20:57.104 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-last-ref-held-by-io 2024-05-04T12:20:57.287 INFO:teuthology.orchestra.run.smithi052.stdout:Building vfs 2024-05-04T12:20:57.295 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] vfstest 2024-05-04T12:21:01.615 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] mount-idmapped 2024-05-04T12:21:02.064 INFO:teuthology.orchestra.run.smithi052.stdout:Building m4 2024-05-04T12:21:02.069 INFO:teuthology.orchestra.run.smithi052.stdout:Building common 2024-05-04T12:21:02.073 INFO:teuthology.orchestra.run.smithi052.stdout:Building tests 2024-05-04T12:21:02.079 INFO:teuthology.orchestra.run.smithi052.stdout:Building btrfs 2024-05-04T12:21:02.088 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/btrfs/group.list 2024-05-04T12:21:02.169 INFO:teuthology.orchestra.run.smithi052.stdout:Building ceph 2024-05-04T12:21:02.175 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/ceph/group.list 2024-05-04T12:21:02.193 INFO:teuthology.orchestra.run.smithi052.stdout:Building cifs 2024-05-04T12:21:02.198 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/cifs/group.list 2024-05-04T12:21:02.213 INFO:teuthology.orchestra.run.smithi052.stdout:Building ext4 2024-05-04T12:21:02.219 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/ext4/group.list 2024-05-04T12:21:02.263 INFO:teuthology.orchestra.run.smithi052.stdout:Building f2fs 2024-05-04T12:21:02.270 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/f2fs/group.list 2024-05-04T12:21:02.290 INFO:teuthology.orchestra.run.smithi052.stdout:Building generic 2024-05-04T12:21:02.297 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/generic/group.list 2024-05-04T12:21:02.418 INFO:teuthology.orchestra.run.smithi052.stdout:Building nfs 2024-05-04T12:21:02.423 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/nfs/group.list 2024-05-04T12:21:02.442 INFO:teuthology.orchestra.run.smithi052.stdout:Building ocfs2 2024-05-04T12:21:02.447 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/ocfs2/group.list 2024-05-04T12:21:02.465 INFO:teuthology.orchestra.run.smithi052.stdout:Building overlay 2024-05-04T12:21:02.472 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/overlay/group.list 2024-05-04T12:21:02.525 INFO:teuthology.orchestra.run.smithi052.stdout:Building perf 2024-05-04T12:21:02.531 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/perf/group.list 2024-05-04T12:21:02.547 INFO:teuthology.orchestra.run.smithi052.stdout:Building selftest 2024-05-04T12:21:02.553 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/selftest/group.list 2024-05-04T12:21:02.570 INFO:teuthology.orchestra.run.smithi052.stdout:Building shared 2024-05-04T12:21:02.577 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/shared/group.list 2024-05-04T12:21:02.599 INFO:teuthology.orchestra.run.smithi052.stdout:Building tmpfs 2024-05-04T12:21:02.606 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/tmpfs/group.list 2024-05-04T12:21:02.622 INFO:teuthology.orchestra.run.smithi052.stdout:Building udf 2024-05-04T12:21:02.629 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/udf/group.list 2024-05-04T12:21:02.643 INFO:teuthology.orchestra.run.smithi052.stdout:Building xfs 2024-05-04T12:21:02.650 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/xfs/group.list 2024-05-04T12:21:02.764 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo make install) 2024-05-04T12:21:02.791 INFO:teuthology.orchestra.run.smithi052.stdout:Building include 2024-05-04T12:21:02.796 INFO:teuthology.orchestra.run.smithi052.stdout:Building lib 2024-05-04T12:21:02.800 INFO:teuthology.orchestra.run.smithi052.stdout:Building ltp 2024-05-04T12:21:02.805 INFO:teuthology.orchestra.run.smithi052.stdout:Building src 2024-05-04T12:21:02.847 INFO:teuthology.orchestra.run.smithi052.stdout:Building log-writes 2024-05-04T12:21:02.851 INFO:teuthology.orchestra.run.smithi052.stdout:Building perf 2024-05-04T12:21:02.855 INFO:teuthology.orchestra.run.smithi052.stdout:Building aio-dio-regress 2024-05-04T12:21:02.859 INFO:teuthology.orchestra.run.smithi052.stdout:Building vfs 2024-05-04T12:21:02.864 INFO:teuthology.orchestra.run.smithi052.stdout:Building m4 2024-05-04T12:21:02.868 INFO:teuthology.orchestra.run.smithi052.stdout:Building common 2024-05-04T12:21:02.873 INFO:teuthology.orchestra.run.smithi052.stdout:Building tests 2024-05-04T12:21:02.879 INFO:teuthology.orchestra.run.smithi052.stdout:Building btrfs 2024-05-04T12:21:02.885 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/btrfs/group.list 2024-05-04T12:21:02.950 INFO:teuthology.orchestra.run.smithi052.stdout:Building ceph 2024-05-04T12:21:02.956 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/ceph/group.list 2024-05-04T12:21:02.972 INFO:teuthology.orchestra.run.smithi052.stdout:Building cifs 2024-05-04T12:21:02.977 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/cifs/group.list 2024-05-04T12:21:02.993 INFO:teuthology.orchestra.run.smithi052.stdout:Building ext4 2024-05-04T12:21:02.999 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/ext4/group.list 2024-05-04T12:21:03.042 INFO:teuthology.orchestra.run.smithi052.stdout:Building f2fs 2024-05-04T12:21:03.047 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/f2fs/group.list 2024-05-04T12:21:03.067 INFO:teuthology.orchestra.run.smithi052.stdout:Building generic 2024-05-04T12:21:03.074 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/generic/group.list 2024-05-04T12:21:03.180 INFO:teuthology.orchestra.run.smithi052.stdout:Building nfs 2024-05-04T12:21:03.187 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/nfs/group.list 2024-05-04T12:21:03.202 INFO:teuthology.orchestra.run.smithi052.stdout:Building ocfs2 2024-05-04T12:21:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/ocfs2/group.list 2024-05-04T12:21:03.222 INFO:teuthology.orchestra.run.smithi052.stdout:Building overlay 2024-05-04T12:21:03.228 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/overlay/group.list 2024-05-04T12:21:03.274 INFO:teuthology.orchestra.run.smithi052.stdout:Building perf 2024-05-04T12:21:03.279 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/perf/group.list 2024-05-04T12:21:03.291 INFO:teuthology.orchestra.run.smithi052.stdout:Building selftest 2024-05-04T12:21:03.297 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/selftest/group.list 2024-05-04T12:21:03.311 INFO:teuthology.orchestra.run.smithi052.stdout:Building shared 2024-05-04T12:21:03.317 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/shared/group.list 2024-05-04T12:21:03.336 INFO:teuthology.orchestra.run.smithi052.stdout:Building tmpfs 2024-05-04T12:21:03.342 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/tmpfs/group.list 2024-05-04T12:21:03.357 INFO:teuthology.orchestra.run.smithi052.stdout:Building udf 2024-05-04T12:21:03.363 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/udf/group.list 2024-05-04T12:21:03.376 INFO:teuthology.orchestra.run.smithi052.stdout:Building xfs 2024-05-04T12:21:03.384 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/xfs/group.list 2024-05-04T12:21:03.491 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-05-04T12:21:03.494 INFO:teuthology.orchestra.run.smithi052.stdout:gmake[1]: Nothing to be done for 'install'. 2024-05-04T12:21:03.494 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-05-04T12:21:03.497 INFO:teuthology.orchestra.run.smithi052.stdout:gmake[1]: Nothing to be done for 'install'. 2024-05-04T12:21:03.497 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-05-04T12:21:03.500 INFO:teuthology.orchestra.run.smithi052.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-05-04T12:21:03.512 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-05-04T12:21:03.600 INFO:teuthology.orchestra.run.smithi052.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-05-04T12:21:03.611 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-05-04T12:21:03.631 INFO:teuthology.orchestra.run.smithi052.stdout:Building log-writes 2024-05-04T12:21:03.635 INFO:teuthology.orchestra.run.smithi052.stdout:Building perf 2024-05-04T12:21:03.638 INFO:teuthology.orchestra.run.smithi052.stdout:Building aio-dio-regress 2024-05-04T12:21:03.645 INFO:teuthology.orchestra.run.smithi052.stdout:Building vfs 2024-05-04T12:21:03.647 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake -C log-writes install 2024-05-04T12:21:03.649 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-05-04T12:21:03.659 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-05-04T12:21:03.670 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake -C perf install 2024-05-04T12:21:03.672 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-05-04T12:21:03.682 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-05-04T12:21:03.707 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-05-04T12:21:03.710 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-05-04T12:21:03.720 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-05-04T12:21:03.772 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake -C vfs install 2024-05-04T12:21:03.774 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-05-04T12:21:03.784 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-05-04T12:21:03.799 INFO:teuthology.orchestra.run.smithi052.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-05-04T12:21:03.808 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault fiemap-tester t_stripealign open_by_handle t_mmap_dio uring_read_fault xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-05-04T12:21:05.448 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-05-04T12:21:05.583 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-05-04T12:21:05.616 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-05-04T12:21:05.619 INFO:teuthology.orchestra.run.smithi052.stdout:gmake[1]: Nothing to be done for 'install'. 2024-05-04T12:21:05.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-05-04T12:21:05.622 INFO:teuthology.orchestra.run.smithi052.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-05-04T12:21:05.633 INFO:teuthology.orchestra.run.smithi052.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-05-04T12:21:05.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-05-04T12:21:05.793 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/btrfs/ install 2024-05-04T12:21:05.798 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/btrfs/group.list 2024-05-04T12:21:05.868 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-05-04T12:21:05.878 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 320 330 /var/lib/xfstests/tests/btrfs 2024-05-04T12:21:06.900 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-05-04T12:21:06.914 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 320.out 330.out /var/lib/xfstests/tests/btrfs 2024-05-04T12:21:08.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/ceph/ install 2024-05-04T12:21:08.008 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/ceph/group.list 2024-05-04T12:21:08.026 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-05-04T12:21:08.038 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-05-04T12:21:08.062 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-05-04T12:21:08.074 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-05-04T12:21:08.098 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/cifs/ install 2024-05-04T12:21:08.102 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/cifs/group.list 2024-05-04T12:21:08.118 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-05-04T12:21:08.130 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-05-04T12:21:08.142 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-05-04T12:21:08.154 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-05-04T12:21:08.165 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/ext4/ install 2024-05-04T12:21:08.169 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/ext4/group.list 2024-05-04T12:21:08.213 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-05-04T12:21:08.224 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-05-04T12:21:08.448 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-05-04T12:21:08.462 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-05-04T12:21:08.708 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/f2fs/ install 2024-05-04T12:21:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/f2fs/group.list 2024-05-04T12:21:08.735 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-05-04T12:21:08.749 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-05-04T12:21:08.762 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-05-04T12:21:08.774 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-05-04T12:21:08.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/generic/ install 2024-05-04T12:21:08.794 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/generic/group.list 2024-05-04T12:21:08.917 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-05-04T12:21:08.929 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 741 742 743 /var/lib/xfstests/tests/generic 2024-05-04T12:21:11.421 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-05-04T12:21:11.434 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 741.out 742.out 743.out /var/lib/xfstests/tests/generic 2024-05-04T12:21:14.020 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/nfs/ install 2024-05-04T12:21:14.025 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/nfs/group.list 2024-05-04T12:21:14.043 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-05-04T12:21:14.056 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-05-04T12:21:14.069 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-05-04T12:21:14.081 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-05-04T12:21:14.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/ocfs2/ install 2024-05-04T12:21:14.097 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/ocfs2/group.list 2024-05-04T12:21:14.112 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-05-04T12:21:14.124 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-05-04T12:21:14.135 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-05-04T12:21:14.146 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-05-04T12:21:14.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/overlay/ install 2024-05-04T12:21:14.161 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/overlay/group.list 2024-05-04T12:21:14.210 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-05-04T12:21:14.221 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-05-04T12:21:14.565 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-05-04T12:21:14.578 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-05-04T12:21:14.921 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/perf/ install 2024-05-04T12:21:14.925 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/perf/group.list 2024-05-04T12:21:14.939 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-05-04T12:21:14.950 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-05-04T12:21:14.962 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-05-04T12:21:14.973 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-05-04T12:21:14.984 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/selftest/ install 2024-05-04T12:21:14.988 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/selftest/group.list 2024-05-04T12:21:15.003 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-05-04T12:21:15.014 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-05-04T12:21:15.040 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-05-04T12:21:15.052 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-05-04T12:21:15.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/shared/ install 2024-05-04T12:21:15.084 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/shared/group.list 2024-05-04T12:21:15.106 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-05-04T12:21:15.117 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-05-04T12:21:15.134 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-05-04T12:21:15.145 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-05-04T12:21:15.161 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/tmpfs/ install 2024-05-04T12:21:15.164 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/tmpfs/group.list 2024-05-04T12:21:15.180 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-05-04T12:21:15.190 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-05-04T12:21:15.201 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-05-04T12:21:15.212 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-05-04T12:21:15.223 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/udf/ install 2024-05-04T12:21:15.227 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/udf/group.list 2024-05-04T12:21:15.239 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-05-04T12:21:15.249 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-05-04T12:21:15.259 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-05-04T12:21:15.269 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-05-04T12:21:15.280 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/xfs/ install 2024-05-04T12:21:15.284 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/xfs/group.list 2024-05-04T12:21:15.398 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-05-04T12:21:15.409 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-05-04T12:21:17.653 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-05-04T12:21:17.666 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-05-04T12:21:20.065 INFO:teuthology.orchestra.run.smithi052.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-05-04T12:21:20.077 INFO:teuthology.orchestra.run.smithi052.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-05-04T12:21:20.089 INFO:teuthology.orchestra.run.smithi052.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-05-04T12:21:20.102 INFO:teuthology.orchestra.run.smithi052.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-05-04T12:21:20.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T12:21:20.117 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec ls tests/generic) 2024-05-04T12:21:20.166 INFO:teuthology.orchestra.run.smithi052.stdout:001 2024-05-04T12:21:20.166 INFO:teuthology.orchestra.run.smithi052.stdout:001.out 2024-05-04T12:21:20.166 INFO:teuthology.orchestra.run.smithi052.stdout:002 2024-05-04T12:21:20.166 INFO:teuthology.orchestra.run.smithi052.stdout:002.out 2024-05-04T12:21:20.166 INFO:teuthology.orchestra.run.smithi052.stdout:003 2024-05-04T12:21:20.166 INFO:teuthology.orchestra.run.smithi052.stdout:003.out 2024-05-04T12:21:20.166 INFO:teuthology.orchestra.run.smithi052.stdout:004 2024-05-04T12:21:20.166 INFO:teuthology.orchestra.run.smithi052.stdout:004.out 2024-05-04T12:21:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:005 2024-05-04T12:21:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:005.out 2024-05-04T12:21:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:006 2024-05-04T12:21:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:006.out 2024-05-04T12:21:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:007 2024-05-04T12:21:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:007.out 2024-05-04T12:21:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:008 2024-05-04T12:21:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:008.out 2024-05-04T12:21:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:009 2024-05-04T12:21:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:009.out 2024-05-04T12:21:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:010 2024-05-04T12:21:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:010.out 2024-05-04T12:21:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:011 2024-05-04T12:21:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:011.out 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:012 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:012.out 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:013 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:013.out 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:014 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:014.out 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:015 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:015.out 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:016 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:016.out 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:017 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:017.out 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:018 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:018.out 2024-05-04T12:21:20.168 INFO:teuthology.orchestra.run.smithi052.stdout:019 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:019.out 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:020 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:020.out 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:021 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:021.out 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:022 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:022.out 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:023 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:023.out 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:024 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:024.out 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:025 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:025.out 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:026 2024-05-04T12:21:20.169 INFO:teuthology.orchestra.run.smithi052.stdout:026.out 2024-05-04T12:21:20.170 INFO:teuthology.orchestra.run.smithi052.stdout:027 2024-05-04T12:21:20.170 INFO:teuthology.orchestra.run.smithi052.stdout:027.out 2024-05-04T12:21:20.170 INFO:teuthology.orchestra.run.smithi052.stdout:028 2024-05-04T12:21:20.170 INFO:teuthology.orchestra.run.smithi052.stdout:028.out 2024-05-04T12:21:20.170 INFO:teuthology.orchestra.run.smithi052.stdout:029 2024-05-04T12:21:20.170 INFO:teuthology.orchestra.run.smithi052.stdout:029.out 2024-05-04T12:21:20.170 INFO:teuthology.orchestra.run.smithi052.stdout:030 2024-05-04T12:21:20.170 INFO:teuthology.orchestra.run.smithi052.stdout:030.out 2024-05-04T12:21:20.170 INFO:teuthology.orchestra.run.smithi052.stdout:031 2024-05-04T12:21:20.170 INFO:teuthology.orchestra.run.smithi052.stdout:031.out 2024-05-04T12:21:20.170 INFO:teuthology.orchestra.run.smithi052.stdout:032 2024-05-04T12:21:20.171 INFO:teuthology.orchestra.run.smithi052.stdout:032.out 2024-05-04T12:21:20.171 INFO:teuthology.orchestra.run.smithi052.stdout:033 2024-05-04T12:21:20.171 INFO:teuthology.orchestra.run.smithi052.stdout:033.out 2024-05-04T12:21:20.171 INFO:teuthology.orchestra.run.smithi052.stdout:034 2024-05-04T12:21:20.171 INFO:teuthology.orchestra.run.smithi052.stdout:034.out 2024-05-04T12:21:20.171 INFO:teuthology.orchestra.run.smithi052.stdout:035 2024-05-04T12:21:20.171 INFO:teuthology.orchestra.run.smithi052.stdout:035.cfg 2024-05-04T12:21:20.171 INFO:teuthology.orchestra.run.smithi052.stdout:035.out.default 2024-05-04T12:21:20.171 INFO:teuthology.orchestra.run.smithi052.stdout:035.out.nfs 2024-05-04T12:21:20.171 INFO:teuthology.orchestra.run.smithi052.stdout:036 2024-05-04T12:21:20.171 INFO:teuthology.orchestra.run.smithi052.stdout:036.out 2024-05-04T12:21:20.171 INFO:teuthology.orchestra.run.smithi052.stdout:037 2024-05-04T12:21:20.171 INFO:teuthology.orchestra.run.smithi052.stdout:037.out 2024-05-04T12:21:20.172 INFO:teuthology.orchestra.run.smithi052.stdout:038 2024-05-04T12:21:20.172 INFO:teuthology.orchestra.run.smithi052.stdout:038.out 2024-05-04T12:21:20.172 INFO:teuthology.orchestra.run.smithi052.stdout:039 2024-05-04T12:21:20.172 INFO:teuthology.orchestra.run.smithi052.stdout:039.out 2024-05-04T12:21:20.172 INFO:teuthology.orchestra.run.smithi052.stdout:040 2024-05-04T12:21:20.172 INFO:teuthology.orchestra.run.smithi052.stdout:040.out 2024-05-04T12:21:20.172 INFO:teuthology.orchestra.run.smithi052.stdout:041 2024-05-04T12:21:20.172 INFO:teuthology.orchestra.run.smithi052.stdout:041.out 2024-05-04T12:21:20.172 INFO:teuthology.orchestra.run.smithi052.stdout:042 2024-05-04T12:21:20.172 INFO:teuthology.orchestra.run.smithi052.stdout:042.out 2024-05-04T12:21:20.172 INFO:teuthology.orchestra.run.smithi052.stdout:043 2024-05-04T12:21:20.172 INFO:teuthology.orchestra.run.smithi052.stdout:043.out 2024-05-04T12:21:20.172 INFO:teuthology.orchestra.run.smithi052.stdout:044 2024-05-04T12:21:20.172 INFO:teuthology.orchestra.run.smithi052.stdout:044.out 2024-05-04T12:21:20.173 INFO:teuthology.orchestra.run.smithi052.stdout:045 2024-05-04T12:21:20.173 INFO:teuthology.orchestra.run.smithi052.stdout:045.out 2024-05-04T12:21:20.173 INFO:teuthology.orchestra.run.smithi052.stdout:046 2024-05-04T12:21:20.173 INFO:teuthology.orchestra.run.smithi052.stdout:046.out 2024-05-04T12:21:20.173 INFO:teuthology.orchestra.run.smithi052.stdout:047 2024-05-04T12:21:20.173 INFO:teuthology.orchestra.run.smithi052.stdout:047.out 2024-05-04T12:21:20.173 INFO:teuthology.orchestra.run.smithi052.stdout:048 2024-05-04T12:21:20.173 INFO:teuthology.orchestra.run.smithi052.stdout:048.out 2024-05-04T12:21:20.173 INFO:teuthology.orchestra.run.smithi052.stdout:049 2024-05-04T12:21:20.173 INFO:teuthology.orchestra.run.smithi052.stdout:049.out 2024-05-04T12:21:20.173 INFO:teuthology.orchestra.run.smithi052.stdout:050 2024-05-04T12:21:20.173 INFO:teuthology.orchestra.run.smithi052.stdout:050.cfg 2024-05-04T12:21:20.174 INFO:teuthology.orchestra.run.smithi052.stdout:050.out.default 2024-05-04T12:21:20.174 INFO:teuthology.orchestra.run.smithi052.stdout:050.out.nojournal 2024-05-04T12:21:20.174 INFO:teuthology.orchestra.run.smithi052.stdout:050.out.xfsquota 2024-05-04T12:21:20.174 INFO:teuthology.orchestra.run.smithi052.stdout:051 2024-05-04T12:21:20.174 INFO:teuthology.orchestra.run.smithi052.stdout:051.out 2024-05-04T12:21:20.174 INFO:teuthology.orchestra.run.smithi052.stdout:052 2024-05-04T12:21:20.174 INFO:teuthology.orchestra.run.smithi052.stdout:052.out 2024-05-04T12:21:20.174 INFO:teuthology.orchestra.run.smithi052.stdout:053 2024-05-04T12:21:20.174 INFO:teuthology.orchestra.run.smithi052.stdout:053.out 2024-05-04T12:21:20.174 INFO:teuthology.orchestra.run.smithi052.stdout:054 2024-05-04T12:21:20.174 INFO:teuthology.orchestra.run.smithi052.stdout:054.out 2024-05-04T12:21:20.174 INFO:teuthology.orchestra.run.smithi052.stdout:055 2024-05-04T12:21:20.174 INFO:teuthology.orchestra.run.smithi052.stdout:055.out 2024-05-04T12:21:20.174 INFO:teuthology.orchestra.run.smithi052.stdout:056 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:056.out 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:057 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:057.out 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:058 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:058.out 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:059 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:059.out 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:060 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:060.out 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:061 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:061.out 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:062 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:062.out 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:063 2024-05-04T12:21:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:063.out 2024-05-04T12:21:20.176 INFO:teuthology.orchestra.run.smithi052.stdout:064 2024-05-04T12:21:20.176 INFO:teuthology.orchestra.run.smithi052.stdout:064.out 2024-05-04T12:21:20.176 INFO:teuthology.orchestra.run.smithi052.stdout:065 2024-05-04T12:21:20.176 INFO:teuthology.orchestra.run.smithi052.stdout:065.out 2024-05-04T12:21:20.176 INFO:teuthology.orchestra.run.smithi052.stdout:066 2024-05-04T12:21:20.176 INFO:teuthology.orchestra.run.smithi052.stdout:066.out 2024-05-04T12:21:20.176 INFO:teuthology.orchestra.run.smithi052.stdout:067 2024-05-04T12:21:20.176 INFO:teuthology.orchestra.run.smithi052.stdout:067.out 2024-05-04T12:21:20.176 INFO:teuthology.orchestra.run.smithi052.stdout:068 2024-05-04T12:21:20.176 INFO:teuthology.orchestra.run.smithi052.stdout:068.out 2024-05-04T12:21:20.176 INFO:teuthology.orchestra.run.smithi052.stdout:069 2024-05-04T12:21:20.176 INFO:teuthology.orchestra.run.smithi052.stdout:069.out 2024-05-04T12:21:20.176 INFO:teuthology.orchestra.run.smithi052.stdout:070 2024-05-04T12:21:20.176 INFO:teuthology.orchestra.run.smithi052.stdout:070.out 2024-05-04T12:21:20.177 INFO:teuthology.orchestra.run.smithi052.stdout:071 2024-05-04T12:21:20.177 INFO:teuthology.orchestra.run.smithi052.stdout:071.out 2024-05-04T12:21:20.177 INFO:teuthology.orchestra.run.smithi052.stdout:072 2024-05-04T12:21:20.177 INFO:teuthology.orchestra.run.smithi052.stdout:072.out 2024-05-04T12:21:20.177 INFO:teuthology.orchestra.run.smithi052.stdout:073 2024-05-04T12:21:20.177 INFO:teuthology.orchestra.run.smithi052.stdout:073.out 2024-05-04T12:21:20.177 INFO:teuthology.orchestra.run.smithi052.stdout:074 2024-05-04T12:21:20.177 INFO:teuthology.orchestra.run.smithi052.stdout:074.out 2024-05-04T12:21:20.177 INFO:teuthology.orchestra.run.smithi052.stdout:075 2024-05-04T12:21:20.177 INFO:teuthology.orchestra.run.smithi052.stdout:075.out 2024-05-04T12:21:20.177 INFO:teuthology.orchestra.run.smithi052.stdout:076 2024-05-04T12:21:20.177 INFO:teuthology.orchestra.run.smithi052.stdout:076.out 2024-05-04T12:21:20.177 INFO:teuthology.orchestra.run.smithi052.stdout:077 2024-05-04T12:21:20.177 INFO:teuthology.orchestra.run.smithi052.stdout:077.out 2024-05-04T12:21:20.178 INFO:teuthology.orchestra.run.smithi052.stdout:078 2024-05-04T12:21:20.178 INFO:teuthology.orchestra.run.smithi052.stdout:078.out 2024-05-04T12:21:20.178 INFO:teuthology.orchestra.run.smithi052.stdout:079 2024-05-04T12:21:20.178 INFO:teuthology.orchestra.run.smithi052.stdout:079.out 2024-05-04T12:21:20.178 INFO:teuthology.orchestra.run.smithi052.stdout:080 2024-05-04T12:21:20.178 INFO:teuthology.orchestra.run.smithi052.stdout:080.out 2024-05-04T12:21:20.178 INFO:teuthology.orchestra.run.smithi052.stdout:081 2024-05-04T12:21:20.178 INFO:teuthology.orchestra.run.smithi052.stdout:081.out 2024-05-04T12:21:20.178 INFO:teuthology.orchestra.run.smithi052.stdout:082 2024-05-04T12:21:20.178 INFO:teuthology.orchestra.run.smithi052.stdout:082.out 2024-05-04T12:21:20.178 INFO:teuthology.orchestra.run.smithi052.stdout:083 2024-05-04T12:21:20.178 INFO:teuthology.orchestra.run.smithi052.stdout:083.out 2024-05-04T12:21:20.178 INFO:teuthology.orchestra.run.smithi052.stdout:084 2024-05-04T12:21:20.178 INFO:teuthology.orchestra.run.smithi052.stdout:084.out 2024-05-04T12:21:20.179 INFO:teuthology.orchestra.run.smithi052.stdout:085 2024-05-04T12:21:20.179 INFO:teuthology.orchestra.run.smithi052.stdout:085.out 2024-05-04T12:21:20.179 INFO:teuthology.orchestra.run.smithi052.stdout:086 2024-05-04T12:21:20.179 INFO:teuthology.orchestra.run.smithi052.stdout:086.out 2024-05-04T12:21:20.179 INFO:teuthology.orchestra.run.smithi052.stdout:087 2024-05-04T12:21:20.179 INFO:teuthology.orchestra.run.smithi052.stdout:087.out 2024-05-04T12:21:20.179 INFO:teuthology.orchestra.run.smithi052.stdout:088 2024-05-04T12:21:20.179 INFO:teuthology.orchestra.run.smithi052.stdout:088.out 2024-05-04T12:21:20.179 INFO:teuthology.orchestra.run.smithi052.stdout:089 2024-05-04T12:21:20.179 INFO:teuthology.orchestra.run.smithi052.stdout:089.out 2024-05-04T12:21:20.179 INFO:teuthology.orchestra.run.smithi052.stdout:090 2024-05-04T12:21:20.179 INFO:teuthology.orchestra.run.smithi052.stdout:090.out 2024-05-04T12:21:20.179 INFO:teuthology.orchestra.run.smithi052.stdout:091 2024-05-04T12:21:20.179 INFO:teuthology.orchestra.run.smithi052.stdout:091.out 2024-05-04T12:21:20.180 INFO:teuthology.orchestra.run.smithi052.stdout:092 2024-05-04T12:21:20.180 INFO:teuthology.orchestra.run.smithi052.stdout:092.out 2024-05-04T12:21:20.180 INFO:teuthology.orchestra.run.smithi052.stdout:093 2024-05-04T12:21:20.180 INFO:teuthology.orchestra.run.smithi052.stdout:093.out 2024-05-04T12:21:20.180 INFO:teuthology.orchestra.run.smithi052.stdout:094 2024-05-04T12:21:20.180 INFO:teuthology.orchestra.run.smithi052.stdout:094.out 2024-05-04T12:21:20.180 INFO:teuthology.orchestra.run.smithi052.stdout:095 2024-05-04T12:21:20.180 INFO:teuthology.orchestra.run.smithi052.stdout:095.out 2024-05-04T12:21:20.180 INFO:teuthology.orchestra.run.smithi052.stdout:096 2024-05-04T12:21:20.180 INFO:teuthology.orchestra.run.smithi052.stdout:096.out 2024-05-04T12:21:20.180 INFO:teuthology.orchestra.run.smithi052.stdout:097 2024-05-04T12:21:20.180 INFO:teuthology.orchestra.run.smithi052.stdout:097.out 2024-05-04T12:21:20.180 INFO:teuthology.orchestra.run.smithi052.stdout:098 2024-05-04T12:21:20.180 INFO:teuthology.orchestra.run.smithi052.stdout:098.out 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:099 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:099.out 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:100 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:100.out 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:101 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:101.out 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:102 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:102.out 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:103 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:103.out 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:104 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:104.out 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:105 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:105.out 2024-05-04T12:21:20.181 INFO:teuthology.orchestra.run.smithi052.stdout:106 2024-05-04T12:21:20.182 INFO:teuthology.orchestra.run.smithi052.stdout:106.out 2024-05-04T12:21:20.182 INFO:teuthology.orchestra.run.smithi052.stdout:107 2024-05-04T12:21:20.182 INFO:teuthology.orchestra.run.smithi052.stdout:107.out 2024-05-04T12:21:20.182 INFO:teuthology.orchestra.run.smithi052.stdout:108 2024-05-04T12:21:20.182 INFO:teuthology.orchestra.run.smithi052.stdout:108.out 2024-05-04T12:21:20.182 INFO:teuthology.orchestra.run.smithi052.stdout:109 2024-05-04T12:21:20.182 INFO:teuthology.orchestra.run.smithi052.stdout:109.out 2024-05-04T12:21:20.182 INFO:teuthology.orchestra.run.smithi052.stdout:110 2024-05-04T12:21:20.182 INFO:teuthology.orchestra.run.smithi052.stdout:110.out 2024-05-04T12:21:20.182 INFO:teuthology.orchestra.run.smithi052.stdout:111 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:111.out 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:112 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:112.out 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:113 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:113.out 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:114 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:114.out 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:115 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:115.out 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:116 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:116.out 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:117 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:117.out 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:118 2024-05-04T12:21:20.183 INFO:teuthology.orchestra.run.smithi052.stdout:118.out 2024-05-04T12:21:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:119 2024-05-04T12:21:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:119.out 2024-05-04T12:21:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:120 2024-05-04T12:21:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:120.out 2024-05-04T12:21:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:121 2024-05-04T12:21:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:121.out 2024-05-04T12:21:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:122 2024-05-04T12:21:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:122.out 2024-05-04T12:21:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:123 2024-05-04T12:21:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:123.out 2024-05-04T12:21:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:124 2024-05-04T12:21:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:124.out 2024-05-04T12:21:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:125 2024-05-04T12:21:20.185 INFO:teuthology.orchestra.run.smithi052.stdout:125.out 2024-05-04T12:21:20.185 INFO:teuthology.orchestra.run.smithi052.stdout:126 2024-05-04T12:21:20.185 INFO:teuthology.orchestra.run.smithi052.stdout:126.out 2024-05-04T12:21:20.185 INFO:teuthology.orchestra.run.smithi052.stdout:127 2024-05-04T12:21:20.185 INFO:teuthology.orchestra.run.smithi052.stdout:127.out 2024-05-04T12:21:20.185 INFO:teuthology.orchestra.run.smithi052.stdout:128 2024-05-04T12:21:20.185 INFO:teuthology.orchestra.run.smithi052.stdout:128.out 2024-05-04T12:21:20.185 INFO:teuthology.orchestra.run.smithi052.stdout:129 2024-05-04T12:21:20.185 INFO:teuthology.orchestra.run.smithi052.stdout:129.out 2024-05-04T12:21:20.185 INFO:teuthology.orchestra.run.smithi052.stdout:130 2024-05-04T12:21:20.185 INFO:teuthology.orchestra.run.smithi052.stdout:130.out 2024-05-04T12:21:20.185 INFO:teuthology.orchestra.run.smithi052.stdout:131 2024-05-04T12:21:20.185 INFO:teuthology.orchestra.run.smithi052.stdout:131.out 2024-05-04T12:21:20.186 INFO:teuthology.orchestra.run.smithi052.stdout:132 2024-05-04T12:21:20.186 INFO:teuthology.orchestra.run.smithi052.stdout:132.out 2024-05-04T12:21:20.186 INFO:teuthology.orchestra.run.smithi052.stdout:133 2024-05-04T12:21:20.186 INFO:teuthology.orchestra.run.smithi052.stdout:133.out 2024-05-04T12:21:20.186 INFO:teuthology.orchestra.run.smithi052.stdout:134 2024-05-04T12:21:20.186 INFO:teuthology.orchestra.run.smithi052.stdout:134.out 2024-05-04T12:21:20.186 INFO:teuthology.orchestra.run.smithi052.stdout:135 2024-05-04T12:21:20.186 INFO:teuthology.orchestra.run.smithi052.stdout:135.out 2024-05-04T12:21:20.186 INFO:teuthology.orchestra.run.smithi052.stdout:136 2024-05-04T12:21:20.186 INFO:teuthology.orchestra.run.smithi052.stdout:136.out 2024-05-04T12:21:20.186 INFO:teuthology.orchestra.run.smithi052.stdout:137 2024-05-04T12:21:20.186 INFO:teuthology.orchestra.run.smithi052.stdout:137.out 2024-05-04T12:21:20.186 INFO:teuthology.orchestra.run.smithi052.stdout:138 2024-05-04T12:21:20.186 INFO:teuthology.orchestra.run.smithi052.stdout:138.out 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:139 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:139.out 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:140 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:140.out 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:141 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:141.out 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:142 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:142.out 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:143 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:143.out 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:144 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:144.out 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:145 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:145.out 2024-05-04T12:21:20.187 INFO:teuthology.orchestra.run.smithi052.stdout:146 2024-05-04T12:21:20.188 INFO:teuthology.orchestra.run.smithi052.stdout:146.out 2024-05-04T12:21:20.188 INFO:teuthology.orchestra.run.smithi052.stdout:147 2024-05-04T12:21:20.188 INFO:teuthology.orchestra.run.smithi052.stdout:147.out 2024-05-04T12:21:20.188 INFO:teuthology.orchestra.run.smithi052.stdout:148 2024-05-04T12:21:20.188 INFO:teuthology.orchestra.run.smithi052.stdout:148.out 2024-05-04T12:21:20.188 INFO:teuthology.orchestra.run.smithi052.stdout:149 2024-05-04T12:21:20.188 INFO:teuthology.orchestra.run.smithi052.stdout:149.out 2024-05-04T12:21:20.188 INFO:teuthology.orchestra.run.smithi052.stdout:150 2024-05-04T12:21:20.188 INFO:teuthology.orchestra.run.smithi052.stdout:150.out 2024-05-04T12:21:20.188 INFO:teuthology.orchestra.run.smithi052.stdout:151 2024-05-04T12:21:20.188 INFO:teuthology.orchestra.run.smithi052.stdout:151.out 2024-05-04T12:21:20.188 INFO:teuthology.orchestra.run.smithi052.stdout:152 2024-05-04T12:21:20.188 INFO:teuthology.orchestra.run.smithi052.stdout:152.out 2024-05-04T12:21:20.188 INFO:teuthology.orchestra.run.smithi052.stdout:153 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:153.out 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:154 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:154.out 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:155 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:155.out 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:156 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:156.out 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:157 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:157.out 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:158 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:158.out 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:159 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:159.out 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:160 2024-05-04T12:21:20.189 INFO:teuthology.orchestra.run.smithi052.stdout:160.out 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:161 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:161.out 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:162 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:162.out 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:163 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:163.out 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:164 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:164.out 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:165 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:165.out 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:166 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:166.out 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:167 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:167.out 2024-05-04T12:21:20.190 INFO:teuthology.orchestra.run.smithi052.stdout:168 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:168.out 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:169 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:169.out 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:170 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:170.out 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:171 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:171.out 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:172 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:172.out 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:173 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:173.out 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:174 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:174.out 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:175 2024-05-04T12:21:20.191 INFO:teuthology.orchestra.run.smithi052.stdout:175.out 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:176 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:176.out 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:177 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:177.out 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:178 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:178.out 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:179 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:179.out 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:180 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:180.out 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:181 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:181.out 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:182 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:182.out 2024-05-04T12:21:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:183 2024-05-04T12:21:20.193 INFO:teuthology.orchestra.run.smithi052.stdout:183.out 2024-05-04T12:21:20.193 INFO:teuthology.orchestra.run.smithi052.stdout:184 2024-05-04T12:21:20.193 INFO:teuthology.orchestra.run.smithi052.stdout:184.out 2024-05-04T12:21:20.193 INFO:teuthology.orchestra.run.smithi052.stdout:185 2024-05-04T12:21:20.193 INFO:teuthology.orchestra.run.smithi052.stdout:185.out 2024-05-04T12:21:20.193 INFO:teuthology.orchestra.run.smithi052.stdout:186 2024-05-04T12:21:20.193 INFO:teuthology.orchestra.run.smithi052.stdout:186.out 2024-05-04T12:21:20.193 INFO:teuthology.orchestra.run.smithi052.stdout:187 2024-05-04T12:21:20.193 INFO:teuthology.orchestra.run.smithi052.stdout:187.out 2024-05-04T12:21:20.193 INFO:teuthology.orchestra.run.smithi052.stdout:188 2024-05-04T12:21:20.193 INFO:teuthology.orchestra.run.smithi052.stdout:188.out 2024-05-04T12:21:20.193 INFO:teuthology.orchestra.run.smithi052.stdout:189 2024-05-04T12:21:20.193 INFO:teuthology.orchestra.run.smithi052.stdout:189.out 2024-05-04T12:21:20.194 INFO:teuthology.orchestra.run.smithi052.stdout:190 2024-05-04T12:21:20.194 INFO:teuthology.orchestra.run.smithi052.stdout:190.out 2024-05-04T12:21:20.194 INFO:teuthology.orchestra.run.smithi052.stdout:191 2024-05-04T12:21:20.194 INFO:teuthology.orchestra.run.smithi052.stdout:191.out 2024-05-04T12:21:20.194 INFO:teuthology.orchestra.run.smithi052.stdout:192 2024-05-04T12:21:20.194 INFO:teuthology.orchestra.run.smithi052.stdout:192.out 2024-05-04T12:21:20.194 INFO:teuthology.orchestra.run.smithi052.stdout:193 2024-05-04T12:21:20.194 INFO:teuthology.orchestra.run.smithi052.stdout:193.out 2024-05-04T12:21:20.194 INFO:teuthology.orchestra.run.smithi052.stdout:194 2024-05-04T12:21:20.194 INFO:teuthology.orchestra.run.smithi052.stdout:194.out 2024-05-04T12:21:20.194 INFO:teuthology.orchestra.run.smithi052.stdout:195 2024-05-04T12:21:20.194 INFO:teuthology.orchestra.run.smithi052.stdout:195.out 2024-05-04T12:21:20.194 INFO:teuthology.orchestra.run.smithi052.stdout:196 2024-05-04T12:21:20.194 INFO:teuthology.orchestra.run.smithi052.stdout:196.out 2024-05-04T12:21:20.195 INFO:teuthology.orchestra.run.smithi052.stdout:197 2024-05-04T12:21:20.195 INFO:teuthology.orchestra.run.smithi052.stdout:197.out 2024-05-04T12:21:20.195 INFO:teuthology.orchestra.run.smithi052.stdout:198 2024-05-04T12:21:20.195 INFO:teuthology.orchestra.run.smithi052.stdout:198.out 2024-05-04T12:21:20.195 INFO:teuthology.orchestra.run.smithi052.stdout:199 2024-05-04T12:21:20.195 INFO:teuthology.orchestra.run.smithi052.stdout:199.out 2024-05-04T12:21:20.195 INFO:teuthology.orchestra.run.smithi052.stdout:200 2024-05-04T12:21:20.195 INFO:teuthology.orchestra.run.smithi052.stdout:200.out 2024-05-04T12:21:20.195 INFO:teuthology.orchestra.run.smithi052.stdout:201 2024-05-04T12:21:20.195 INFO:teuthology.orchestra.run.smithi052.stdout:201.out 2024-05-04T12:21:20.195 INFO:teuthology.orchestra.run.smithi052.stdout:202 2024-05-04T12:21:20.195 INFO:teuthology.orchestra.run.smithi052.stdout:202.out 2024-05-04T12:21:20.195 INFO:teuthology.orchestra.run.smithi052.stdout:203 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:203.out 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:204 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:204.out 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:205 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:205.out 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:206 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:206.out 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:207 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:207.out 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:208 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:208.out 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:209 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:209.out 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:210 2024-05-04T12:21:20.196 INFO:teuthology.orchestra.run.smithi052.stdout:210.out 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:211 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:211.out 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:212 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:212.out 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:213 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:213.out 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:214 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:214.out 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:215 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:215.out 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:216 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:216.out 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:217 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:217.out 2024-05-04T12:21:20.197 INFO:teuthology.orchestra.run.smithi052.stdout:218 2024-05-04T12:21:20.198 INFO:teuthology.orchestra.run.smithi052.stdout:218.out 2024-05-04T12:21:20.198 INFO:teuthology.orchestra.run.smithi052.stdout:219 2024-05-04T12:21:20.198 INFO:teuthology.orchestra.run.smithi052.stdout:219.out 2024-05-04T12:21:20.199 INFO:teuthology.orchestra.run.smithi052.stdout:220 2024-05-04T12:21:20.199 INFO:teuthology.orchestra.run.smithi052.stdout:220.out 2024-05-04T12:21:20.199 INFO:teuthology.orchestra.run.smithi052.stdout:221 2024-05-04T12:21:20.199 INFO:teuthology.orchestra.run.smithi052.stdout:221.out 2024-05-04T12:21:20.199 INFO:teuthology.orchestra.run.smithi052.stdout:222 2024-05-04T12:21:20.200 INFO:teuthology.orchestra.run.smithi052.stdout:222.out 2024-05-04T12:21:20.200 INFO:teuthology.orchestra.run.smithi052.stdout:223 2024-05-04T12:21:20.200 INFO:teuthology.orchestra.run.smithi052.stdout:223.out 2024-05-04T12:21:20.201 INFO:teuthology.orchestra.run.smithi052.stdout:224 2024-05-04T12:21:20.201 INFO:teuthology.orchestra.run.smithi052.stdout:224.out 2024-05-04T12:21:20.201 INFO:teuthology.orchestra.run.smithi052.stdout:225 2024-05-04T12:21:20.201 INFO:teuthology.orchestra.run.smithi052.stdout:225.out 2024-05-04T12:21:20.201 INFO:teuthology.orchestra.run.smithi052.stdout:226 2024-05-04T12:21:20.201 INFO:teuthology.orchestra.run.smithi052.stdout:226.out 2024-05-04T12:21:20.201 INFO:teuthology.orchestra.run.smithi052.stdout:227 2024-05-04T12:21:20.201 INFO:teuthology.orchestra.run.smithi052.stdout:227.out 2024-05-04T12:21:20.202 INFO:teuthology.orchestra.run.smithi052.stdout:228 2024-05-04T12:21:20.202 INFO:teuthology.orchestra.run.smithi052.stdout:228.out 2024-05-04T12:21:20.202 INFO:teuthology.orchestra.run.smithi052.stdout:229 2024-05-04T12:21:20.202 INFO:teuthology.orchestra.run.smithi052.stdout:229.out 2024-05-04T12:21:20.202 INFO:teuthology.orchestra.run.smithi052.stdout:230 2024-05-04T12:21:20.202 INFO:teuthology.orchestra.run.smithi052.stdout:230.out 2024-05-04T12:21:20.202 INFO:teuthology.orchestra.run.smithi052.stdout:231 2024-05-04T12:21:20.202 INFO:teuthology.orchestra.run.smithi052.stdout:231.out 2024-05-04T12:21:20.202 INFO:teuthology.orchestra.run.smithi052.stdout:232 2024-05-04T12:21:20.202 INFO:teuthology.orchestra.run.smithi052.stdout:232.out 2024-05-04T12:21:20.202 INFO:teuthology.orchestra.run.smithi052.stdout:233 2024-05-04T12:21:20.202 INFO:teuthology.orchestra.run.smithi052.stdout:233.out 2024-05-04T12:21:20.202 INFO:teuthology.orchestra.run.smithi052.stdout:234 2024-05-04T12:21:20.203 INFO:teuthology.orchestra.run.smithi052.stdout:234.out 2024-05-04T12:21:20.203 INFO:teuthology.orchestra.run.smithi052.stdout:235 2024-05-04T12:21:20.203 INFO:teuthology.orchestra.run.smithi052.stdout:235.out 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:236 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:236.out 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:237 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:237.out 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:238 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:238.out 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:239 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:239.out 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:240 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:240.out 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:241 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:241.out 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:242 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:242.out 2024-05-04T12:21:20.204 INFO:teuthology.orchestra.run.smithi052.stdout:243 2024-05-04T12:21:20.205 INFO:teuthology.orchestra.run.smithi052.stdout:243.out 2024-05-04T12:21:20.205 INFO:teuthology.orchestra.run.smithi052.stdout:244 2024-05-04T12:21:20.205 INFO:teuthology.orchestra.run.smithi052.stdout:244.out 2024-05-04T12:21:20.205 INFO:teuthology.orchestra.run.smithi052.stdout:245 2024-05-04T12:21:20.205 INFO:teuthology.orchestra.run.smithi052.stdout:245.out 2024-05-04T12:21:20.205 INFO:teuthology.orchestra.run.smithi052.stdout:246 2024-05-04T12:21:20.205 INFO:teuthology.orchestra.run.smithi052.stdout:246.out 2024-05-04T12:21:20.205 INFO:teuthology.orchestra.run.smithi052.stdout:247 2024-05-04T12:21:20.205 INFO:teuthology.orchestra.run.smithi052.stdout:247.out 2024-05-04T12:21:20.206 INFO:teuthology.orchestra.run.smithi052.stdout:248 2024-05-04T12:21:20.206 INFO:teuthology.orchestra.run.smithi052.stdout:248.out 2024-05-04T12:21:20.206 INFO:teuthology.orchestra.run.smithi052.stdout:249 2024-05-04T12:21:20.206 INFO:teuthology.orchestra.run.smithi052.stdout:249.out 2024-05-04T12:21:20.206 INFO:teuthology.orchestra.run.smithi052.stdout:250 2024-05-04T12:21:20.206 INFO:teuthology.orchestra.run.smithi052.stdout:250.out 2024-05-04T12:21:20.206 INFO:teuthology.orchestra.run.smithi052.stdout:251 2024-05-04T12:21:20.206 INFO:teuthology.orchestra.run.smithi052.stdout:251.out 2024-05-04T12:21:20.206 INFO:teuthology.orchestra.run.smithi052.stdout:252 2024-05-04T12:21:20.206 INFO:teuthology.orchestra.run.smithi052.stdout:252.out 2024-05-04T12:21:20.206 INFO:teuthology.orchestra.run.smithi052.stdout:253 2024-05-04T12:21:20.206 INFO:teuthology.orchestra.run.smithi052.stdout:253.out 2024-05-04T12:21:20.206 INFO:teuthology.orchestra.run.smithi052.stdout:254 2024-05-04T12:21:20.206 INFO:teuthology.orchestra.run.smithi052.stdout:254.out 2024-05-04T12:21:20.207 INFO:teuthology.orchestra.run.smithi052.stdout:255 2024-05-04T12:21:20.207 INFO:teuthology.orchestra.run.smithi052.stdout:255.out 2024-05-04T12:21:20.207 INFO:teuthology.orchestra.run.smithi052.stdout:256 2024-05-04T12:21:20.207 INFO:teuthology.orchestra.run.smithi052.stdout:256.out 2024-05-04T12:21:20.207 INFO:teuthology.orchestra.run.smithi052.stdout:257 2024-05-04T12:21:20.207 INFO:teuthology.orchestra.run.smithi052.stdout:257.out 2024-05-04T12:21:20.207 INFO:teuthology.orchestra.run.smithi052.stdout:258 2024-05-04T12:21:20.207 INFO:teuthology.orchestra.run.smithi052.stdout:258.out 2024-05-04T12:21:20.207 INFO:teuthology.orchestra.run.smithi052.stdout:259 2024-05-04T12:21:20.207 INFO:teuthology.orchestra.run.smithi052.stdout:259.out 2024-05-04T12:21:20.207 INFO:teuthology.orchestra.run.smithi052.stdout:260 2024-05-04T12:21:20.207 INFO:teuthology.orchestra.run.smithi052.stdout:260.out 2024-05-04T12:21:20.207 INFO:teuthology.orchestra.run.smithi052.stdout:261 2024-05-04T12:21:20.208 INFO:teuthology.orchestra.run.smithi052.stdout:261.out 2024-05-04T12:21:20.208 INFO:teuthology.orchestra.run.smithi052.stdout:262 2024-05-04T12:21:20.208 INFO:teuthology.orchestra.run.smithi052.stdout:262.out 2024-05-04T12:21:20.208 INFO:teuthology.orchestra.run.smithi052.stdout:263 2024-05-04T12:21:20.208 INFO:teuthology.orchestra.run.smithi052.stdout:263.out 2024-05-04T12:21:20.208 INFO:teuthology.orchestra.run.smithi052.stdout:264 2024-05-04T12:21:20.208 INFO:teuthology.orchestra.run.smithi052.stdout:264.out 2024-05-04T12:21:20.208 INFO:teuthology.orchestra.run.smithi052.stdout:265 2024-05-04T12:21:20.208 INFO:teuthology.orchestra.run.smithi052.stdout:265.out 2024-05-04T12:21:20.208 INFO:teuthology.orchestra.run.smithi052.stdout:266 2024-05-04T12:21:20.208 INFO:teuthology.orchestra.run.smithi052.stdout:266.out 2024-05-04T12:21:20.208 INFO:teuthology.orchestra.run.smithi052.stdout:267 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:267.out 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:268 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:268.out 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:269 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:269.out 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:270 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:270.out 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:271 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:271.out 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:272 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:272.out 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:273 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:273.out 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:274 2024-05-04T12:21:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:274.out 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:275 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:275.out 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:276 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:276.out 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:277 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:277.out 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:278 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:278.out 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:279 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:279.out 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:280 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:280.out 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:281 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:281.out 2024-05-04T12:21:20.210 INFO:teuthology.orchestra.run.smithi052.stdout:282 2024-05-04T12:21:20.211 INFO:teuthology.orchestra.run.smithi052.stdout:282.out 2024-05-04T12:21:20.211 INFO:teuthology.orchestra.run.smithi052.stdout:283 2024-05-04T12:21:20.211 INFO:teuthology.orchestra.run.smithi052.stdout:283.out 2024-05-04T12:21:20.211 INFO:teuthology.orchestra.run.smithi052.stdout:284 2024-05-04T12:21:20.211 INFO:teuthology.orchestra.run.smithi052.stdout:284.out 2024-05-04T12:21:20.211 INFO:teuthology.orchestra.run.smithi052.stdout:285 2024-05-04T12:21:20.211 INFO:teuthology.orchestra.run.smithi052.stdout:285.out 2024-05-04T12:21:20.211 INFO:teuthology.orchestra.run.smithi052.stdout:286 2024-05-04T12:21:20.211 INFO:teuthology.orchestra.run.smithi052.stdout:286.out 2024-05-04T12:21:20.211 INFO:teuthology.orchestra.run.smithi052.stdout:287 2024-05-04T12:21:20.212 INFO:teuthology.orchestra.run.smithi052.stdout:287.out 2024-05-04T12:21:20.212 INFO:teuthology.orchestra.run.smithi052.stdout:288 2024-05-04T12:21:20.212 INFO:teuthology.orchestra.run.smithi052.stdout:288.out 2024-05-04T12:21:20.212 INFO:teuthology.orchestra.run.smithi052.stdout:289 2024-05-04T12:21:20.212 INFO:teuthology.orchestra.run.smithi052.stdout:289.out 2024-05-04T12:21:20.212 INFO:teuthology.orchestra.run.smithi052.stdout:290 2024-05-04T12:21:20.212 INFO:teuthology.orchestra.run.smithi052.stdout:290.out 2024-05-04T12:21:20.212 INFO:teuthology.orchestra.run.smithi052.stdout:291 2024-05-04T12:21:20.212 INFO:teuthology.orchestra.run.smithi052.stdout:291.out 2024-05-04T12:21:20.212 INFO:teuthology.orchestra.run.smithi052.stdout:292 2024-05-04T12:21:20.212 INFO:teuthology.orchestra.run.smithi052.stdout:292.out 2024-05-04T12:21:20.212 INFO:teuthology.orchestra.run.smithi052.stdout:293 2024-05-04T12:21:20.212 INFO:teuthology.orchestra.run.smithi052.stdout:293.out 2024-05-04T12:21:20.212 INFO:teuthology.orchestra.run.smithi052.stdout:294 2024-05-04T12:21:20.213 INFO:teuthology.orchestra.run.smithi052.stdout:294.out 2024-05-04T12:21:20.213 INFO:teuthology.orchestra.run.smithi052.stdout:295 2024-05-04T12:21:20.213 INFO:teuthology.orchestra.run.smithi052.stdout:295.out 2024-05-04T12:21:20.213 INFO:teuthology.orchestra.run.smithi052.stdout:296 2024-05-04T12:21:20.213 INFO:teuthology.orchestra.run.smithi052.stdout:296.out 2024-05-04T12:21:20.213 INFO:teuthology.orchestra.run.smithi052.stdout:297 2024-05-04T12:21:20.213 INFO:teuthology.orchestra.run.smithi052.stdout:297.out 2024-05-04T12:21:20.213 INFO:teuthology.orchestra.run.smithi052.stdout:298 2024-05-04T12:21:20.213 INFO:teuthology.orchestra.run.smithi052.stdout:298.out 2024-05-04T12:21:20.213 INFO:teuthology.orchestra.run.smithi052.stdout:299 2024-05-04T12:21:20.213 INFO:teuthology.orchestra.run.smithi052.stdout:299.out 2024-05-04T12:21:20.213 INFO:teuthology.orchestra.run.smithi052.stdout:300 2024-05-04T12:21:20.213 INFO:teuthology.orchestra.run.smithi052.stdout:300.out 2024-05-04T12:21:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:301 2024-05-04T12:21:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:301.out 2024-05-04T12:21:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:302 2024-05-04T12:21:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:302.out 2024-05-04T12:21:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:303 2024-05-04T12:21:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:303.out 2024-05-04T12:21:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:304 2024-05-04T12:21:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:304.out 2024-05-04T12:21:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:305 2024-05-04T12:21:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:305.out 2024-05-04T12:21:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:306 2024-05-04T12:21:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:306.out 2024-05-04T12:21:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:307 2024-05-04T12:21:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:307.out 2024-05-04T12:21:20.215 INFO:teuthology.orchestra.run.smithi052.stdout:308 2024-05-04T12:21:20.215 INFO:teuthology.orchestra.run.smithi052.stdout:308.out 2024-05-04T12:21:20.215 INFO:teuthology.orchestra.run.smithi052.stdout:309 2024-05-04T12:21:20.215 INFO:teuthology.orchestra.run.smithi052.stdout:309.out 2024-05-04T12:21:20.215 INFO:teuthology.orchestra.run.smithi052.stdout:310 2024-05-04T12:21:20.215 INFO:teuthology.orchestra.run.smithi052.stdout:310.out 2024-05-04T12:21:20.215 INFO:teuthology.orchestra.run.smithi052.stdout:311 2024-05-04T12:21:20.215 INFO:teuthology.orchestra.run.smithi052.stdout:311.out 2024-05-04T12:21:20.215 INFO:teuthology.orchestra.run.smithi052.stdout:312 2024-05-04T12:21:20.215 INFO:teuthology.orchestra.run.smithi052.stdout:312.out 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:313 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:313.out 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:314 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:314.out 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:315 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:315.out 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:316 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:316.out 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:317 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:317.out 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:318 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:318.out 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:319 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:319.out 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:320 2024-05-04T12:21:20.216 INFO:teuthology.orchestra.run.smithi052.stdout:320.out 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:321 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:321.out 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:322 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:322.out 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:323 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:323.out 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:324 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:324.out 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:325 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:325.out 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:326 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:326.out 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:327 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:327.out 2024-05-04T12:21:20.217 INFO:teuthology.orchestra.run.smithi052.stdout:328 2024-05-04T12:21:20.218 INFO:teuthology.orchestra.run.smithi052.stdout:328.out 2024-05-04T12:21:20.218 INFO:teuthology.orchestra.run.smithi052.stdout:329 2024-05-04T12:21:20.218 INFO:teuthology.orchestra.run.smithi052.stdout:329.out 2024-05-04T12:21:20.218 INFO:teuthology.orchestra.run.smithi052.stdout:330 2024-05-04T12:21:20.218 INFO:teuthology.orchestra.run.smithi052.stdout:330.out 2024-05-04T12:21:20.218 INFO:teuthology.orchestra.run.smithi052.stdout:331 2024-05-04T12:21:20.218 INFO:teuthology.orchestra.run.smithi052.stdout:331.out 2024-05-04T12:21:20.218 INFO:teuthology.orchestra.run.smithi052.stdout:332 2024-05-04T12:21:20.218 INFO:teuthology.orchestra.run.smithi052.stdout:332.out 2024-05-04T12:21:20.218 INFO:teuthology.orchestra.run.smithi052.stdout:333 2024-05-04T12:21:20.218 INFO:teuthology.orchestra.run.smithi052.stdout:333.out 2024-05-04T12:21:20.218 INFO:teuthology.orchestra.run.smithi052.stdout:334 2024-05-04T12:21:20.218 INFO:teuthology.orchestra.run.smithi052.stdout:334.out 2024-05-04T12:21:20.218 INFO:teuthology.orchestra.run.smithi052.stdout:335 2024-05-04T12:21:20.219 INFO:teuthology.orchestra.run.smithi052.stdout:335.out 2024-05-04T12:21:20.219 INFO:teuthology.orchestra.run.smithi052.stdout:336 2024-05-04T12:21:20.219 INFO:teuthology.orchestra.run.smithi052.stdout:336.out 2024-05-04T12:21:20.219 INFO:teuthology.orchestra.run.smithi052.stdout:337 2024-05-04T12:21:20.219 INFO:teuthology.orchestra.run.smithi052.stdout:337.out 2024-05-04T12:21:20.219 INFO:teuthology.orchestra.run.smithi052.stdout:338 2024-05-04T12:21:20.219 INFO:teuthology.orchestra.run.smithi052.stdout:338.out 2024-05-04T12:21:20.219 INFO:teuthology.orchestra.run.smithi052.stdout:339 2024-05-04T12:21:20.219 INFO:teuthology.orchestra.run.smithi052.stdout:339.out 2024-05-04T12:21:20.219 INFO:teuthology.orchestra.run.smithi052.stdout:340 2024-05-04T12:21:20.219 INFO:teuthology.orchestra.run.smithi052.stdout:340.out 2024-05-04T12:21:20.219 INFO:teuthology.orchestra.run.smithi052.stdout:341 2024-05-04T12:21:20.220 INFO:teuthology.orchestra.run.smithi052.stdout:341.out 2024-05-04T12:21:20.220 INFO:teuthology.orchestra.run.smithi052.stdout:342 2024-05-04T12:21:20.220 INFO:teuthology.orchestra.run.smithi052.stdout:342.out 2024-05-04T12:21:20.220 INFO:teuthology.orchestra.run.smithi052.stdout:343 2024-05-04T12:21:20.221 INFO:teuthology.orchestra.run.smithi052.stdout:343.out 2024-05-04T12:21:20.222 INFO:teuthology.orchestra.run.smithi052.stdout:344 2024-05-04T12:21:20.222 INFO:teuthology.orchestra.run.smithi052.stdout:344.out 2024-05-04T12:21:20.222 INFO:teuthology.orchestra.run.smithi052.stdout:345 2024-05-04T12:21:20.222 INFO:teuthology.orchestra.run.smithi052.stdout:345.out 2024-05-04T12:21:20.222 INFO:teuthology.orchestra.run.smithi052.stdout:346 2024-05-04T12:21:20.222 INFO:teuthology.orchestra.run.smithi052.stdout:346.out 2024-05-04T12:21:20.222 INFO:teuthology.orchestra.run.smithi052.stdout:347 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:347.out 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:348 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:348.out 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:349 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:349.out 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:350 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:350.out 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:351 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:351.out 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:352 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:352.out 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:353 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:353.out 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:354 2024-05-04T12:21:20.223 INFO:teuthology.orchestra.run.smithi052.stdout:354.out 2024-05-04T12:21:20.224 INFO:teuthology.orchestra.run.smithi052.stdout:355 2024-05-04T12:21:20.224 INFO:teuthology.orchestra.run.smithi052.stdout:355.out 2024-05-04T12:21:20.224 INFO:teuthology.orchestra.run.smithi052.stdout:356 2024-05-04T12:21:20.224 INFO:teuthology.orchestra.run.smithi052.stdout:356.out 2024-05-04T12:21:20.224 INFO:teuthology.orchestra.run.smithi052.stdout:357 2024-05-04T12:21:20.224 INFO:teuthology.orchestra.run.smithi052.stdout:357.out 2024-05-04T12:21:20.224 INFO:teuthology.orchestra.run.smithi052.stdout:358 2024-05-04T12:21:20.224 INFO:teuthology.orchestra.run.smithi052.stdout:358.out 2024-05-04T12:21:20.224 INFO:teuthology.orchestra.run.smithi052.stdout:359 2024-05-04T12:21:20.224 INFO:teuthology.orchestra.run.smithi052.stdout:359.out 2024-05-04T12:21:20.224 INFO:teuthology.orchestra.run.smithi052.stdout:360 2024-05-04T12:21:20.224 INFO:teuthology.orchestra.run.smithi052.stdout:360.out 2024-05-04T12:21:20.224 INFO:teuthology.orchestra.run.smithi052.stdout:361 2024-05-04T12:21:20.224 INFO:teuthology.orchestra.run.smithi052.stdout:361.out 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:362 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:362.out 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:363 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:363.out 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:364 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:364.out 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:365 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:365.out 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:366 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:366.out 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:367 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:367.out 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:368 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:368.out 2024-05-04T12:21:20.225 INFO:teuthology.orchestra.run.smithi052.stdout:369 2024-05-04T12:21:20.226 INFO:teuthology.orchestra.run.smithi052.stdout:369.out 2024-05-04T12:21:20.226 INFO:teuthology.orchestra.run.smithi052.stdout:370 2024-05-04T12:21:20.226 INFO:teuthology.orchestra.run.smithi052.stdout:370.out 2024-05-04T12:21:20.226 INFO:teuthology.orchestra.run.smithi052.stdout:371 2024-05-04T12:21:20.226 INFO:teuthology.orchestra.run.smithi052.stdout:371.out 2024-05-04T12:21:20.226 INFO:teuthology.orchestra.run.smithi052.stdout:372 2024-05-04T12:21:20.226 INFO:teuthology.orchestra.run.smithi052.stdout:372.out 2024-05-04T12:21:20.226 INFO:teuthology.orchestra.run.smithi052.stdout:373 2024-05-04T12:21:20.226 INFO:teuthology.orchestra.run.smithi052.stdout:373.out 2024-05-04T12:21:20.226 INFO:teuthology.orchestra.run.smithi052.stdout:374 2024-05-04T12:21:20.226 INFO:teuthology.orchestra.run.smithi052.stdout:374.out 2024-05-04T12:21:20.226 INFO:teuthology.orchestra.run.smithi052.stdout:375 2024-05-04T12:21:20.226 INFO:teuthology.orchestra.run.smithi052.stdout:375.out 2024-05-04T12:21:20.226 INFO:teuthology.orchestra.run.smithi052.stdout:376 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:376.out 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:377 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:377.out 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:378 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:378.out 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:379 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:379.out 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:380 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:380.out 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:381 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:381.out 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:382 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:382.out 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:383 2024-05-04T12:21:20.227 INFO:teuthology.orchestra.run.smithi052.stdout:383.out 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:384 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:384.out 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:385 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:385.out 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:386 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:386.out 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:387 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:387.out 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:388 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:388.out 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:389 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:389.out 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:390 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:390.out 2024-05-04T12:21:20.228 INFO:teuthology.orchestra.run.smithi052.stdout:391 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:391.out 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:392 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:392.out 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:393 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:393.out 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:394 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:394.out 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:395 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:395.out 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:396 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:396.out 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:397 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:397.out 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:398 2024-05-04T12:21:20.229 INFO:teuthology.orchestra.run.smithi052.stdout:398.out 2024-05-04T12:21:20.230 INFO:teuthology.orchestra.run.smithi052.stdout:399 2024-05-04T12:21:20.230 INFO:teuthology.orchestra.run.smithi052.stdout:399.out 2024-05-04T12:21:20.230 INFO:teuthology.orchestra.run.smithi052.stdout:400 2024-05-04T12:21:20.230 INFO:teuthology.orchestra.run.smithi052.stdout:400.out 2024-05-04T12:21:20.230 INFO:teuthology.orchestra.run.smithi052.stdout:401 2024-05-04T12:21:20.230 INFO:teuthology.orchestra.run.smithi052.stdout:401.out 2024-05-04T12:21:20.230 INFO:teuthology.orchestra.run.smithi052.stdout:402 2024-05-04T12:21:20.230 INFO:teuthology.orchestra.run.smithi052.stdout:402.out 2024-05-04T12:21:20.230 INFO:teuthology.orchestra.run.smithi052.stdout:403 2024-05-04T12:21:20.230 INFO:teuthology.orchestra.run.smithi052.stdout:403.out 2024-05-04T12:21:20.230 INFO:teuthology.orchestra.run.smithi052.stdout:404 2024-05-04T12:21:20.230 INFO:teuthology.orchestra.run.smithi052.stdout:404.out 2024-05-04T12:21:20.230 INFO:teuthology.orchestra.run.smithi052.stdout:405 2024-05-04T12:21:20.231 INFO:teuthology.orchestra.run.smithi052.stdout:405.out 2024-05-04T12:21:20.231 INFO:teuthology.orchestra.run.smithi052.stdout:406 2024-05-04T12:21:20.231 INFO:teuthology.orchestra.run.smithi052.stdout:406.out 2024-05-04T12:21:20.231 INFO:teuthology.orchestra.run.smithi052.stdout:407 2024-05-04T12:21:20.231 INFO:teuthology.orchestra.run.smithi052.stdout:407.out 2024-05-04T12:21:20.231 INFO:teuthology.orchestra.run.smithi052.stdout:408 2024-05-04T12:21:20.231 INFO:teuthology.orchestra.run.smithi052.stdout:408.out 2024-05-04T12:21:20.231 INFO:teuthology.orchestra.run.smithi052.stdout:409 2024-05-04T12:21:20.231 INFO:teuthology.orchestra.run.smithi052.stdout:409.out 2024-05-04T12:21:20.231 INFO:teuthology.orchestra.run.smithi052.stdout:410 2024-05-04T12:21:20.232 INFO:teuthology.orchestra.run.smithi052.stdout:410.out 2024-05-04T12:21:20.232 INFO:teuthology.orchestra.run.smithi052.stdout:411 2024-05-04T12:21:20.232 INFO:teuthology.orchestra.run.smithi052.stdout:411.out 2024-05-04T12:21:20.232 INFO:teuthology.orchestra.run.smithi052.stdout:412 2024-05-04T12:21:20.232 INFO:teuthology.orchestra.run.smithi052.stdout:412.out 2024-05-04T12:21:20.232 INFO:teuthology.orchestra.run.smithi052.stdout:413 2024-05-04T12:21:20.232 INFO:teuthology.orchestra.run.smithi052.stdout:413.out 2024-05-04T12:21:20.232 INFO:teuthology.orchestra.run.smithi052.stdout:414 2024-05-04T12:21:20.232 INFO:teuthology.orchestra.run.smithi052.stdout:414.out 2024-05-04T12:21:20.232 INFO:teuthology.orchestra.run.smithi052.stdout:415 2024-05-04T12:21:20.232 INFO:teuthology.orchestra.run.smithi052.stdout:415.out 2024-05-04T12:21:20.232 INFO:teuthology.orchestra.run.smithi052.stdout:416 2024-05-04T12:21:20.232 INFO:teuthology.orchestra.run.smithi052.stdout:416.out 2024-05-04T12:21:20.232 INFO:teuthology.orchestra.run.smithi052.stdout:417 2024-05-04T12:21:20.233 INFO:teuthology.orchestra.run.smithi052.stdout:417.out 2024-05-04T12:21:20.233 INFO:teuthology.orchestra.run.smithi052.stdout:418 2024-05-04T12:21:20.233 INFO:teuthology.orchestra.run.smithi052.stdout:418.out 2024-05-04T12:21:20.233 INFO:teuthology.orchestra.run.smithi052.stdout:419 2024-05-04T12:21:20.233 INFO:teuthology.orchestra.run.smithi052.stdout:419.out 2024-05-04T12:21:20.233 INFO:teuthology.orchestra.run.smithi052.stdout:420 2024-05-04T12:21:20.233 INFO:teuthology.orchestra.run.smithi052.stdout:420.out 2024-05-04T12:21:20.233 INFO:teuthology.orchestra.run.smithi052.stdout:421 2024-05-04T12:21:20.233 INFO:teuthology.orchestra.run.smithi052.stdout:421.out 2024-05-04T12:21:20.233 INFO:teuthology.orchestra.run.smithi052.stdout:422 2024-05-04T12:21:20.233 INFO:teuthology.orchestra.run.smithi052.stdout:422.out 2024-05-04T12:21:20.233 INFO:teuthology.orchestra.run.smithi052.stdout:423 2024-05-04T12:21:20.234 INFO:teuthology.orchestra.run.smithi052.stdout:423.out 2024-05-04T12:21:20.234 INFO:teuthology.orchestra.run.smithi052.stdout:424 2024-05-04T12:21:20.234 INFO:teuthology.orchestra.run.smithi052.stdout:424.out 2024-05-04T12:21:20.234 INFO:teuthology.orchestra.run.smithi052.stdout:425 2024-05-04T12:21:20.234 INFO:teuthology.orchestra.run.smithi052.stdout:425.out 2024-05-04T12:21:20.234 INFO:teuthology.orchestra.run.smithi052.stdout:426 2024-05-04T12:21:20.234 INFO:teuthology.orchestra.run.smithi052.stdout:426.out 2024-05-04T12:21:20.234 INFO:teuthology.orchestra.run.smithi052.stdout:427 2024-05-04T12:21:20.234 INFO:teuthology.orchestra.run.smithi052.stdout:427.out 2024-05-04T12:21:20.234 INFO:teuthology.orchestra.run.smithi052.stdout:428 2024-05-04T12:21:20.234 INFO:teuthology.orchestra.run.smithi052.stdout:428.out 2024-05-04T12:21:20.234 INFO:teuthology.orchestra.run.smithi052.stdout:429 2024-05-04T12:21:20.234 INFO:teuthology.orchestra.run.smithi052.stdout:429.out 2024-05-04T12:21:20.234 INFO:teuthology.orchestra.run.smithi052.stdout:430 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:430.out 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:431 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:431.out 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:432 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:432.out 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:433 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:433.out 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:434 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:434.out 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:435 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:435.out 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:436 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:436.out 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:437 2024-05-04T12:21:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:437.out 2024-05-04T12:21:20.236 INFO:teuthology.orchestra.run.smithi052.stdout:438 2024-05-04T12:21:20.236 INFO:teuthology.orchestra.run.smithi052.stdout:438.out 2024-05-04T12:21:20.236 INFO:teuthology.orchestra.run.smithi052.stdout:439 2024-05-04T12:21:20.236 INFO:teuthology.orchestra.run.smithi052.stdout:439.out 2024-05-04T12:21:20.236 INFO:teuthology.orchestra.run.smithi052.stdout:440 2024-05-04T12:21:20.236 INFO:teuthology.orchestra.run.smithi052.stdout:440.out 2024-05-04T12:21:20.236 INFO:teuthology.orchestra.run.smithi052.stdout:441 2024-05-04T12:21:20.236 INFO:teuthology.orchestra.run.smithi052.stdout:441.out 2024-05-04T12:21:20.236 INFO:teuthology.orchestra.run.smithi052.stdout:442 2024-05-04T12:21:20.236 INFO:teuthology.orchestra.run.smithi052.stdout:442.out 2024-05-04T12:21:20.236 INFO:teuthology.orchestra.run.smithi052.stdout:443 2024-05-04T12:21:20.236 INFO:teuthology.orchestra.run.smithi052.stdout:443.out 2024-05-04T12:21:20.236 INFO:teuthology.orchestra.run.smithi052.stdout:444 2024-05-04T12:21:20.237 INFO:teuthology.orchestra.run.smithi052.stdout:444.out 2024-05-04T12:21:20.237 INFO:teuthology.orchestra.run.smithi052.stdout:445 2024-05-04T12:21:20.237 INFO:teuthology.orchestra.run.smithi052.stdout:445.out 2024-05-04T12:21:20.237 INFO:teuthology.orchestra.run.smithi052.stdout:446 2024-05-04T12:21:20.237 INFO:teuthology.orchestra.run.smithi052.stdout:446.out 2024-05-04T12:21:20.237 INFO:teuthology.orchestra.run.smithi052.stdout:447 2024-05-04T12:21:20.237 INFO:teuthology.orchestra.run.smithi052.stdout:447.out 2024-05-04T12:21:20.237 INFO:teuthology.orchestra.run.smithi052.stdout:448 2024-05-04T12:21:20.237 INFO:teuthology.orchestra.run.smithi052.stdout:448.out 2024-05-04T12:21:20.237 INFO:teuthology.orchestra.run.smithi052.stdout:449 2024-05-04T12:21:20.237 INFO:teuthology.orchestra.run.smithi052.stdout:449.out 2024-05-04T12:21:20.237 INFO:teuthology.orchestra.run.smithi052.stdout:450 2024-05-04T12:21:20.237 INFO:teuthology.orchestra.run.smithi052.stdout:450.out 2024-05-04T12:21:20.237 INFO:teuthology.orchestra.run.smithi052.stdout:451 2024-05-04T12:21:20.238 INFO:teuthology.orchestra.run.smithi052.stdout:451.out 2024-05-04T12:21:20.238 INFO:teuthology.orchestra.run.smithi052.stdout:452 2024-05-04T12:21:20.238 INFO:teuthology.orchestra.run.smithi052.stdout:452.out 2024-05-04T12:21:20.238 INFO:teuthology.orchestra.run.smithi052.stdout:453 2024-05-04T12:21:20.238 INFO:teuthology.orchestra.run.smithi052.stdout:453.out 2024-05-04T12:21:20.238 INFO:teuthology.orchestra.run.smithi052.stdout:454 2024-05-04T12:21:20.238 INFO:teuthology.orchestra.run.smithi052.stdout:454.out 2024-05-04T12:21:20.238 INFO:teuthology.orchestra.run.smithi052.stdout:455 2024-05-04T12:21:20.238 INFO:teuthology.orchestra.run.smithi052.stdout:455.out 2024-05-04T12:21:20.238 INFO:teuthology.orchestra.run.smithi052.stdout:456 2024-05-04T12:21:20.238 INFO:teuthology.orchestra.run.smithi052.stdout:456.out 2024-05-04T12:21:20.238 INFO:teuthology.orchestra.run.smithi052.stdout:457 2024-05-04T12:21:20.238 INFO:teuthology.orchestra.run.smithi052.stdout:457.out 2024-05-04T12:21:20.238 INFO:teuthology.orchestra.run.smithi052.stdout:458 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:458.out 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:459 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:459.out 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:460 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:460.out 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:461 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:461.out 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:462 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:462.out 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:463 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:463.out 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:464 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:464.out 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:465 2024-05-04T12:21:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:465.out 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:466 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:466.out 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:467 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:467.out 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:468 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:468.out 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:469 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:469.out 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:470 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:470.out 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:471 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:471.out 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:472 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:472.out 2024-05-04T12:21:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:473 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:473.out 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:474 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:474.out 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:475 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:475.out 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:476 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:476.out 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:477 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:477.out 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:478 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:478.out 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:479 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:479.out 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:480 2024-05-04T12:21:20.241 INFO:teuthology.orchestra.run.smithi052.stdout:480.out 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:481 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:481.out 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:482 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:482.out 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:483 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:483.out 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:484 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:484.out 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:485 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:485.out 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:486 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:486.out 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:487 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:487.out 2024-05-04T12:21:20.242 INFO:teuthology.orchestra.run.smithi052.stdout:488 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:488.out 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:489 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:489.out 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:490 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:490.out 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:491 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:491.out 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:492 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:492.out 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:493 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:493.out 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:494 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:494.out 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:495 2024-05-04T12:21:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:495.out 2024-05-04T12:21:20.244 INFO:teuthology.orchestra.run.smithi052.stdout:496 2024-05-04T12:21:20.244 INFO:teuthology.orchestra.run.smithi052.stdout:496.out 2024-05-04T12:21:20.244 INFO:teuthology.orchestra.run.smithi052.stdout:497 2024-05-04T12:21:20.244 INFO:teuthology.orchestra.run.smithi052.stdout:497.out 2024-05-04T12:21:20.244 INFO:teuthology.orchestra.run.smithi052.stdout:498 2024-05-04T12:21:20.244 INFO:teuthology.orchestra.run.smithi052.stdout:498.out 2024-05-04T12:21:20.244 INFO:teuthology.orchestra.run.smithi052.stdout:499 2024-05-04T12:21:20.244 INFO:teuthology.orchestra.run.smithi052.stdout:499.out 2024-05-04T12:21:20.244 INFO:teuthology.orchestra.run.smithi052.stdout:500 2024-05-04T12:21:20.244 INFO:teuthology.orchestra.run.smithi052.stdout:500.out 2024-05-04T12:21:20.244 INFO:teuthology.orchestra.run.smithi052.stdout:501 2024-05-04T12:21:20.244 INFO:teuthology.orchestra.run.smithi052.stdout:501.out 2024-05-04T12:21:20.244 INFO:teuthology.orchestra.run.smithi052.stdout:502 2024-05-04T12:21:20.244 INFO:teuthology.orchestra.run.smithi052.stdout:502.out 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:503 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:503.out 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:504 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:504.out 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:505 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:505.out 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:506 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:506.out 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:507 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:507.out 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:508 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:508.out 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:509 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:509.out 2024-05-04T12:21:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:510 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:510.out 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:511 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:511.out 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:512 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:512.out 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:513 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:513.out 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:514 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:514.out 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:515 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:515.out 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:516 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:516.out 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:517 2024-05-04T12:21:20.246 INFO:teuthology.orchestra.run.smithi052.stdout:517.out 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:518 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:518.out 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:519 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:519.out 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:520 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:520.out 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:521 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:521.out 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:522 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:522.out 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:523 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:523.out 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:524 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:524.out 2024-05-04T12:21:20.247 INFO:teuthology.orchestra.run.smithi052.stdout:525 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:525.out 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:526 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:526.out 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:527 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:527.out 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:528 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:528.out 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:529 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:529.out 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:530 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:530.out 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:531 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:531.out 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:532 2024-05-04T12:21:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:532.out 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:533 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:533.out 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:534 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:534.out 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:535 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:535.out 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:536 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:536.out 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:537 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:537.out 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:538 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:538.out 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:539 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:539.out 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:540 2024-05-04T12:21:20.249 INFO:teuthology.orchestra.run.smithi052.stdout:540.out 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:541 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:541.out 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:542 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:542.out 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:543 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:543.out 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:544 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:544.out 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:545 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:545.out 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:546 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:546.out 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:547 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:547.out 2024-05-04T12:21:20.250 INFO:teuthology.orchestra.run.smithi052.stdout:548 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:548.out 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:549 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:549.out 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:550 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:550.out 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:551 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:551.out 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:552 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:552.out 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:553 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:553.out 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:554 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:554.out 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:555 2024-05-04T12:21:20.251 INFO:teuthology.orchestra.run.smithi052.stdout:555.out 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:556 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:556.out 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:557 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:557.out 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:558 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:558.out 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:559 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:559.out 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:560 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:560.out 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:561 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:561.out 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:562 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:562.out 2024-05-04T12:21:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:563 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:563.out 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:564 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:564.out 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:565 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:565.out 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:566 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:566.out 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:567 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:567.out 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:568 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:568.out 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:569 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:569.out 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:570 2024-05-04T12:21:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:570.out 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:571 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:571.out 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:572 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:572.out 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:573 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:573.out 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:574 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:574.out 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:575 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:575.out 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:576 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:576.out 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:577 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:577.out 2024-05-04T12:21:20.254 INFO:teuthology.orchestra.run.smithi052.stdout:578 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:578.out 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:579 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:579.out 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:580 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:580.out 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:581 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:581.out 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:582 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:582.out 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:583 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:583.out 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:584 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:584.out 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:585 2024-05-04T12:21:20.255 INFO:teuthology.orchestra.run.smithi052.stdout:585.out 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:586 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:586.out 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:587 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:587.out 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:588 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:588.out 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:589 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:589.out 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:590 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:590.out 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:591 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:591.out 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:592 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:592.out 2024-05-04T12:21:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:593 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:593.out 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:594 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:594.out 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:595 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:595.out 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:596 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:596.out 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:597 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:597.out 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:598 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:598.out 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:599 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:599.out 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:600 2024-05-04T12:21:20.257 INFO:teuthology.orchestra.run.smithi052.stdout:600.out 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:601 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:601.out 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:602 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:602.out 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:603 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:603.out 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:604 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:604.out 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:605 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:605.out 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:606 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:606.out 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:607 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:607.out 2024-05-04T12:21:20.258 INFO:teuthology.orchestra.run.smithi052.stdout:608 2024-05-04T12:21:20.259 INFO:teuthology.orchestra.run.smithi052.stdout:608.out 2024-05-04T12:21:20.259 INFO:teuthology.orchestra.run.smithi052.stdout:609 2024-05-04T12:21:20.259 INFO:teuthology.orchestra.run.smithi052.stdout:609.out 2024-05-04T12:21:20.259 INFO:teuthology.orchestra.run.smithi052.stdout:610 2024-05-04T12:21:20.259 INFO:teuthology.orchestra.run.smithi052.stdout:610.out 2024-05-04T12:21:20.259 INFO:teuthology.orchestra.run.smithi052.stdout:611 2024-05-04T12:21:20.259 INFO:teuthology.orchestra.run.smithi052.stdout:611.out 2024-05-04T12:21:20.259 INFO:teuthology.orchestra.run.smithi052.stdout:612 2024-05-04T12:21:20.259 INFO:teuthology.orchestra.run.smithi052.stdout:612.out 2024-05-04T12:21:20.259 INFO:teuthology.orchestra.run.smithi052.stdout:613 2024-05-04T12:21:20.259 INFO:teuthology.orchestra.run.smithi052.stdout:613.out 2024-05-04T12:21:20.259 INFO:teuthology.orchestra.run.smithi052.stdout:614 2024-05-04T12:21:20.259 INFO:teuthology.orchestra.run.smithi052.stdout:614.out 2024-05-04T12:21:20.259 INFO:teuthology.orchestra.run.smithi052.stdout:615 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:615.out 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:616 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:616.out 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:617 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:617.out 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:618 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:618.out 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:619 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:619.out 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:620 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:620.out 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:621 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:621.out 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:622 2024-05-04T12:21:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:622.out 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:623 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:623.out 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:624 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:624.out 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:625 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:625.out 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:626 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:626.out 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:627 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:627.out 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:628 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:628.out 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:629 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:629.out 2024-05-04T12:21:20.261 INFO:teuthology.orchestra.run.smithi052.stdout:630 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:630.out 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:631 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:631.out 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:632 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:632.out 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:633 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:633.out 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:634 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:634.out 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:635 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:635.out 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:636 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:636.out 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:637 2024-05-04T12:21:20.262 INFO:teuthology.orchestra.run.smithi052.stdout:637.out 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:638 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:638.out 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:639 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:639.out 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:640 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:640.out 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:641 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:641.out 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:642 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:642.out 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:643 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:643.out 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:644 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:644.out 2024-05-04T12:21:20.263 INFO:teuthology.orchestra.run.smithi052.stdout:645 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:645.out 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:646 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:646.out 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:647 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:647.out 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:648 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:648.out 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:649 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:649.out 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:650 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:650.out 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:651 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:651.out 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:652 2024-05-04T12:21:20.264 INFO:teuthology.orchestra.run.smithi052.stdout:652.out 2024-05-04T12:21:20.265 INFO:teuthology.orchestra.run.smithi052.stdout:653 2024-05-04T12:21:20.265 INFO:teuthology.orchestra.run.smithi052.stdout:653.out 2024-05-04T12:21:20.265 INFO:teuthology.orchestra.run.smithi052.stdout:654 2024-05-04T12:21:20.265 INFO:teuthology.orchestra.run.smithi052.stdout:654.out 2024-05-04T12:21:20.265 INFO:teuthology.orchestra.run.smithi052.stdout:655 2024-05-04T12:21:20.265 INFO:teuthology.orchestra.run.smithi052.stdout:655.out 2024-05-04T12:21:20.265 INFO:teuthology.orchestra.run.smithi052.stdout:656 2024-05-04T12:21:20.265 INFO:teuthology.orchestra.run.smithi052.stdout:656.out 2024-05-04T12:21:20.265 INFO:teuthology.orchestra.run.smithi052.stdout:657 2024-05-04T12:21:20.265 INFO:teuthology.orchestra.run.smithi052.stdout:657.out 2024-05-04T12:21:20.265 INFO:teuthology.orchestra.run.smithi052.stdout:658 2024-05-04T12:21:20.265 INFO:teuthology.orchestra.run.smithi052.stdout:658.out 2024-05-04T12:21:20.265 INFO:teuthology.orchestra.run.smithi052.stdout:659 2024-05-04T12:21:20.265 INFO:teuthology.orchestra.run.smithi052.stdout:659.out 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:660 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:660.out 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:661 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:661.out 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:662 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:662.out 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:663 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:663.out 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:664 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:664.out 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:665 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:665.out 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:666 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:666.out 2024-05-04T12:21:20.266 INFO:teuthology.orchestra.run.smithi052.stdout:667 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:667.out 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:668 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:668.out 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:669 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:669.out 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:670 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:670.out 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:671 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:671.out 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:672 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:672.out 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:673 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:673.out 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:674 2024-05-04T12:21:20.267 INFO:teuthology.orchestra.run.smithi052.stdout:674.out 2024-05-04T12:21:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:675 2024-05-04T12:21:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:675.out 2024-05-04T12:21:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:676 2024-05-04T12:21:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:676.out 2024-05-04T12:21:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:677 2024-05-04T12:21:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:677.out 2024-05-04T12:21:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:678 2024-05-04T12:21:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:678.out 2024-05-04T12:21:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:679 2024-05-04T12:21:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:679.out 2024-05-04T12:21:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:680 2024-05-04T12:21:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:680.out 2024-05-04T12:21:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:681 2024-05-04T12:21:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:681.out 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:682 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:682.out 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:683 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:683.out 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:684 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:684.out 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:685 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:685.out 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:686 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:686.out 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:687 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:687.out 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:688 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:688.out 2024-05-04T12:21:20.269 INFO:teuthology.orchestra.run.smithi052.stdout:689 2024-05-04T12:21:20.270 INFO:teuthology.orchestra.run.smithi052.stdout:689.out 2024-05-04T12:21:20.270 INFO:teuthology.orchestra.run.smithi052.stdout:690 2024-05-04T12:21:20.270 INFO:teuthology.orchestra.run.smithi052.stdout:690.out 2024-05-04T12:21:20.270 INFO:teuthology.orchestra.run.smithi052.stdout:691 2024-05-04T12:21:20.270 INFO:teuthology.orchestra.run.smithi052.stdout:691.out 2024-05-04T12:21:20.270 INFO:teuthology.orchestra.run.smithi052.stdout:692 2024-05-04T12:21:20.270 INFO:teuthology.orchestra.run.smithi052.stdout:692.out 2024-05-04T12:21:20.270 INFO:teuthology.orchestra.run.smithi052.stdout:693 2024-05-04T12:21:20.270 INFO:teuthology.orchestra.run.smithi052.stdout:693.out 2024-05-04T12:21:20.270 INFO:teuthology.orchestra.run.smithi052.stdout:694 2024-05-04T12:21:20.270 INFO:teuthology.orchestra.run.smithi052.stdout:694.out 2024-05-04T12:21:20.270 INFO:teuthology.orchestra.run.smithi052.stdout:695 2024-05-04T12:21:20.270 INFO:teuthology.orchestra.run.smithi052.stdout:695.out 2024-05-04T12:21:20.270 INFO:teuthology.orchestra.run.smithi052.stdout:696 2024-05-04T12:21:20.271 INFO:teuthology.orchestra.run.smithi052.stdout:696.out 2024-05-04T12:21:20.271 INFO:teuthology.orchestra.run.smithi052.stdout:697 2024-05-04T12:21:20.271 INFO:teuthology.orchestra.run.smithi052.stdout:697.out 2024-05-04T12:21:20.271 INFO:teuthology.orchestra.run.smithi052.stdout:698 2024-05-04T12:21:20.272 INFO:teuthology.orchestra.run.smithi052.stdout:698.out 2024-05-04T12:21:20.272 INFO:teuthology.orchestra.run.smithi052.stdout:699 2024-05-04T12:21:20.272 INFO:teuthology.orchestra.run.smithi052.stdout:699.out 2024-05-04T12:21:20.272 INFO:teuthology.orchestra.run.smithi052.stdout:700 2024-05-04T12:21:20.272 INFO:teuthology.orchestra.run.smithi052.stdout:700.out 2024-05-04T12:21:20.272 INFO:teuthology.orchestra.run.smithi052.stdout:701 2024-05-04T12:21:20.272 INFO:teuthology.orchestra.run.smithi052.stdout:701.out 2024-05-04T12:21:20.273 INFO:teuthology.orchestra.run.smithi052.stdout:702 2024-05-04T12:21:20.273 INFO:teuthology.orchestra.run.smithi052.stdout:702.out 2024-05-04T12:21:20.273 INFO:teuthology.orchestra.run.smithi052.stdout:703 2024-05-04T12:21:20.273 INFO:teuthology.orchestra.run.smithi052.stdout:703.out 2024-05-04T12:21:20.273 INFO:teuthology.orchestra.run.smithi052.stdout:704 2024-05-04T12:21:20.273 INFO:teuthology.orchestra.run.smithi052.stdout:704.out 2024-05-04T12:21:20.273 INFO:teuthology.orchestra.run.smithi052.stdout:705 2024-05-04T12:21:20.274 INFO:teuthology.orchestra.run.smithi052.stdout:705.out 2024-05-04T12:21:20.274 INFO:teuthology.orchestra.run.smithi052.stdout:706 2024-05-04T12:21:20.274 INFO:teuthology.orchestra.run.smithi052.stdout:706.out 2024-05-04T12:21:20.274 INFO:teuthology.orchestra.run.smithi052.stdout:707 2024-05-04T12:21:20.274 INFO:teuthology.orchestra.run.smithi052.stdout:707.out 2024-05-04T12:21:20.274 INFO:teuthology.orchestra.run.smithi052.stdout:708 2024-05-04T12:21:20.275 INFO:teuthology.orchestra.run.smithi052.stdout:708.out 2024-05-04T12:21:20.275 INFO:teuthology.orchestra.run.smithi052.stdout:709 2024-05-04T12:21:20.275 INFO:teuthology.orchestra.run.smithi052.stdout:709.out 2024-05-04T12:21:20.275 INFO:teuthology.orchestra.run.smithi052.stdout:710 2024-05-04T12:21:20.275 INFO:teuthology.orchestra.run.smithi052.stdout:710.out 2024-05-04T12:21:20.276 INFO:teuthology.orchestra.run.smithi052.stdout:711 2024-05-04T12:21:20.276 INFO:teuthology.orchestra.run.smithi052.stdout:711.out 2024-05-04T12:21:20.276 INFO:teuthology.orchestra.run.smithi052.stdout:712 2024-05-04T12:21:20.276 INFO:teuthology.orchestra.run.smithi052.stdout:712.out 2024-05-04T12:21:20.276 INFO:teuthology.orchestra.run.smithi052.stdout:713 2024-05-04T12:21:20.276 INFO:teuthology.orchestra.run.smithi052.stdout:713.out 2024-05-04T12:21:20.277 INFO:teuthology.orchestra.run.smithi052.stdout:714 2024-05-04T12:21:20.277 INFO:teuthology.orchestra.run.smithi052.stdout:714.out 2024-05-04T12:21:20.277 INFO:teuthology.orchestra.run.smithi052.stdout:715 2024-05-04T12:21:20.277 INFO:teuthology.orchestra.run.smithi052.stdout:715.out 2024-05-04T12:21:20.277 INFO:teuthology.orchestra.run.smithi052.stdout:716 2024-05-04T12:21:20.277 INFO:teuthology.orchestra.run.smithi052.stdout:716.out 2024-05-04T12:21:20.277 INFO:teuthology.orchestra.run.smithi052.stdout:717 2024-05-04T12:21:20.278 INFO:teuthology.orchestra.run.smithi052.stdout:717.out 2024-05-04T12:21:20.278 INFO:teuthology.orchestra.run.smithi052.stdout:718 2024-05-04T12:21:20.278 INFO:teuthology.orchestra.run.smithi052.stdout:718.out 2024-05-04T12:21:20.278 INFO:teuthology.orchestra.run.smithi052.stdout:719 2024-05-04T12:21:20.278 INFO:teuthology.orchestra.run.smithi052.stdout:719.out 2024-05-04T12:21:20.278 INFO:teuthology.orchestra.run.smithi052.stdout:720 2024-05-04T12:21:20.279 INFO:teuthology.orchestra.run.smithi052.stdout:720.out 2024-05-04T12:21:20.279 INFO:teuthology.orchestra.run.smithi052.stdout:721 2024-05-04T12:21:20.279 INFO:teuthology.orchestra.run.smithi052.stdout:721.out 2024-05-04T12:21:20.279 INFO:teuthology.orchestra.run.smithi052.stdout:722 2024-05-04T12:21:20.279 INFO:teuthology.orchestra.run.smithi052.stdout:722.out 2024-05-04T12:21:20.280 INFO:teuthology.orchestra.run.smithi052.stdout:723 2024-05-04T12:21:20.280 INFO:teuthology.orchestra.run.smithi052.stdout:723.out 2024-05-04T12:21:20.280 INFO:teuthology.orchestra.run.smithi052.stdout:724 2024-05-04T12:21:20.281 INFO:teuthology.orchestra.run.smithi052.stdout:724.out 2024-05-04T12:21:20.281 INFO:teuthology.orchestra.run.smithi052.stdout:725 2024-05-04T12:21:20.281 INFO:teuthology.orchestra.run.smithi052.stdout:725.out 2024-05-04T12:21:20.281 INFO:teuthology.orchestra.run.smithi052.stdout:726 2024-05-04T12:21:20.281 INFO:teuthology.orchestra.run.smithi052.stdout:726.out 2024-05-04T12:21:20.281 INFO:teuthology.orchestra.run.smithi052.stdout:727 2024-05-04T12:21:20.281 INFO:teuthology.orchestra.run.smithi052.stdout:727.out 2024-05-04T12:21:20.282 INFO:teuthology.orchestra.run.smithi052.stdout:728 2024-05-04T12:21:20.282 INFO:teuthology.orchestra.run.smithi052.stdout:728.out 2024-05-04T12:21:20.282 INFO:teuthology.orchestra.run.smithi052.stdout:729 2024-05-04T12:21:20.282 INFO:teuthology.orchestra.run.smithi052.stdout:729.out 2024-05-04T12:21:20.282 INFO:teuthology.orchestra.run.smithi052.stdout:730 2024-05-04T12:21:20.283 INFO:teuthology.orchestra.run.smithi052.stdout:730.out 2024-05-04T12:21:20.283 INFO:teuthology.orchestra.run.smithi052.stdout:731 2024-05-04T12:21:20.283 INFO:teuthology.orchestra.run.smithi052.stdout:731.out 2024-05-04T12:21:20.283 INFO:teuthology.orchestra.run.smithi052.stdout:732 2024-05-04T12:21:20.283 INFO:teuthology.orchestra.run.smithi052.stdout:732.out 2024-05-04T12:21:20.283 INFO:teuthology.orchestra.run.smithi052.stdout:733 2024-05-04T12:21:20.283 INFO:teuthology.orchestra.run.smithi052.stdout:733.out 2024-05-04T12:21:20.284 INFO:teuthology.orchestra.run.smithi052.stdout:734 2024-05-04T12:21:20.284 INFO:teuthology.orchestra.run.smithi052.stdout:734.out 2024-05-04T12:21:20.284 INFO:teuthology.orchestra.run.smithi052.stdout:735 2024-05-04T12:21:20.284 INFO:teuthology.orchestra.run.smithi052.stdout:735.out 2024-05-04T12:21:20.284 INFO:teuthology.orchestra.run.smithi052.stdout:736 2024-05-04T12:21:20.284 INFO:teuthology.orchestra.run.smithi052.stdout:736.out 2024-05-04T12:21:20.285 INFO:teuthology.orchestra.run.smithi052.stdout:737 2024-05-04T12:21:20.285 INFO:teuthology.orchestra.run.smithi052.stdout:737.out 2024-05-04T12:21:20.285 INFO:teuthology.orchestra.run.smithi052.stdout:738 2024-05-04T12:21:20.285 INFO:teuthology.orchestra.run.smithi052.stdout:738.out 2024-05-04T12:21:20.285 INFO:teuthology.orchestra.run.smithi052.stdout:739 2024-05-04T12:21:20.285 INFO:teuthology.orchestra.run.smithi052.stdout:739.out 2024-05-04T12:21:20.285 INFO:teuthology.orchestra.run.smithi052.stdout:741 2024-05-04T12:21:20.286 INFO:teuthology.orchestra.run.smithi052.stdout:741.out 2024-05-04T12:21:20.286 INFO:teuthology.orchestra.run.smithi052.stdout:742 2024-05-04T12:21:20.286 INFO:teuthology.orchestra.run.smithi052.stdout:742.out 2024-05-04T12:21:20.286 INFO:teuthology.orchestra.run.smithi052.stdout:743 2024-05-04T12:21:20.286 INFO:teuthology.orchestra.run.smithi052.stdout:743.out 2024-05-04T12:21:20.286 INFO:teuthology.orchestra.run.smithi052.stdout:group.list 2024-05-04T12:21:20.287 INFO:teuthology.orchestra.run.smithi052.stdout:Makefile 2024-05-04T12:21:20.289 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2024-05-04T12:21:20.543 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:21:20.548 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:21:20.549 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:21:22.905 INFO:teuthology.orchestra.run.smithi052.stdout:generic/001 2s 2024-05-04T12:21:22.911 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/001 2024-05-04T12:21:22.911 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:21:22.914 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:21:22.916 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[37511]: fuse finished with error 0 and tester_r 0 2024-05-04T12:21:22.927 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:21:22.928 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2024-05-04T12:21:23.108 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:21:23.113 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:21:23.113 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:21:23.558 INFO:teuthology.orchestra.run.smithi052.stdout:generic/002 0s 2024-05-04T12:21:23.564 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/002 2024-05-04T12:21:23.564 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:21:23.566 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:21:23.576 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:21:23.576 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2024-05-04T12:21:23.698 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:21:23.700 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:21:23.763 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:21:23.767 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:21:23.767 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:21:23.868 INFO:teuthology.orchestra.run.smithi052.stdout:generic/003 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:21:23.871 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/003 2024-05-04T12:21:23.871 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/003 2024-05-04T12:21:23.871 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:21:23.873 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:21:23.881 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:21:23.882 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2024-05-04T12:21:24.069 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:21:24.073 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:21:24.073 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:21:24.181 INFO:teuthology.orchestra.run.smithi052.stdout:generic/004 [not run] O_TMPFILE is not supported 2024-05-04T12:21:24.184 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/004 2024-05-04T12:21:24.185 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/004 2024-05-04T12:21:24.185 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:21:24.187 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:21:24.195 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:21:24.195 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2024-05-04T12:21:24.339 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:21:24.343 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:21:24.343 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:21:24.822 INFO:teuthology.orchestra.run.smithi052.stdout:generic/005 0s 2024-05-04T12:21:24.827 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/005 2024-05-04T12:21:24.827 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:21:24.829 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:21:24.838 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:21:24.838 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2024-05-04T12:21:25.020 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:21:25.024 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:21:25.024 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:21:53.742 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:21:53.745 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:22:17.864 INFO:teuthology.orchestra.run.smithi052.stdout:generic/006 52s 2024-05-04T12:22:18.010 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/006 2024-05-04T12:22:18.011 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:22:18.015 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:22:18.101 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:22:18.101 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2024-05-04T12:22:18.414 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:22:18.419 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:22:18.419 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:22:24.136 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:22:24.139 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:22:54.436 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:22:54.438 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:23:24.561 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:23:24.564 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:23:37.154 INFO:teuthology.orchestra.run.smithi052.stdout:generic/007 79s 2024-05-04T12:23:37.225 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/007 2024-05-04T12:23:37.225 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:23:37.228 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:23:37.238 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:23:37.238 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2024-05-04T12:23:37.451 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:23:37.455 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:23:37.455 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:23:38.525 INFO:teuthology.orchestra.run.smithi052.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-05-04T12:23:38.530 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/008 2024-05-04T12:23:38.530 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/008 2024-05-04T12:23:38.530 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:23:38.533 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:23:38.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:23:38.543 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2024-05-04T12:23:38.730 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:23:38.735 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:23:38.735 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:23:39.927 INFO:teuthology.orchestra.run.smithi052.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-05-04T12:23:39.933 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/009 2024-05-04T12:23:39.933 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/009 2024-05-04T12:23:39.933 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:23:39.937 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:23:39.948 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:23:39.948 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2024-05-04T12:23:40.145 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:23:40.149 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:23:40.149 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:23:40.280 INFO:teuthology.orchestra.run.smithi052.stdout:generic/010 [not run] /tmp/tmp.BtfdD8Y1eWxfstests-dev/src/dbtest not built 2024-05-04T12:23:40.284 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/010 2024-05-04T12:23:40.284 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/010 2024-05-04T12:23:40.284 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:23:40.287 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:23:40.297 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:23:40.297 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2024-05-04T12:23:40.470 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:23:40.474 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:23:40.474 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:23:54.992 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:23:54.994 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:24:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:generic/011 40s 2024-05-04T12:24:20.286 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/011 2024-05-04T12:24:20.286 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:24:20.292 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:24:20.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:24:20.310 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2024-05-04T12:24:20.525 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:24:20.529 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:24:20.529 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:24:21.660 INFO:teuthology.orchestra.run.smithi052.stdout:generic/012 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-04T12:24:21.663 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/012 2024-05-04T12:24:21.663 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/012 2024-05-04T12:24:21.663 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:24:21.665 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:24:21.673 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:24:21.674 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2024-05-04T12:24:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:24:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:24:21.866 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:24:25.162 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:24:25.165 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:24:55.827 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:24:55.830 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:25:26.861 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:25:26.865 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:25:57.306 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:25:57.309 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:25:57.344 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-04T12:25:57.337+0000 7f6f26d7f640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 84318) UID: 0 2024-05-04T12:25:57.363 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T12:25:57.341+0000 7fbf91e1b640 -1 Fail to open '/proc/84318/cmdline' error = (2) No such file or directory 2024-05-04T12:25:57.363 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T12:25:57.341+0000 7fbf91e1b640 -1 received signal: Hangup from (PID: 84318) UID: 0 2024-05-04T12:26:22.116 INFO:teuthology.orchestra.run.smithi052.stdout:generic/013 121s 2024-05-04T12:26:22.237 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/013 2024-05-04T12:26:22.237 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:22.246 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:22.257 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:22.257 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2024-05-04T12:26:22.639 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:22.645 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:22.645 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:23.096 INFO:teuthology.orchestra.run.smithi052.stdout:generic/014 [not run] Sparse files are not supported or do not work as expected 2024-05-04T12:26:23.105 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/014 2024-05-04T12:26:23.105 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/014 2024-05-04T12:26:23.105 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:23.107 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:23.118 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:23.118 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2024-05-04T12:26:23.579 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:23.597 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:23.597 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:23.785 INFO:teuthology.orchestra.run.smithi052.stdout:generic/015 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:26:23.790 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/015 2024-05-04T12:26:23.790 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/015 2024-05-04T12:26:23.790 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:23.793 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:23.808 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:23.809 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2024-05-04T12:26:24.194 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:24.201 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:24.201 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:25.936 INFO:teuthology.orchestra.run.smithi052.stdout:generic/016 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-04T12:26:25.940 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/016 2024-05-04T12:26:25.940 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/016 2024-05-04T12:26:25.940 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:25.945 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:26.098 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:26.099 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2024-05-04T12:26:26.447 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:26.453 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:26.453 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:26.619 INFO:teuthology.orchestra.run.smithi052.stdout:generic/017 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:26:26.625 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/017 2024-05-04T12:26:26.625 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/017 2024-05-04T12:26:26.625 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:26.627 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:26.651 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:26.651 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2024-05-04T12:26:26.981 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:26.988 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:26.988 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:27.157 INFO:teuthology.orchestra.run.smithi052.stdout:generic/018 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:26:27.163 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/018 2024-05-04T12:26:27.163 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/018 2024-05-04T12:26:27.163 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:27.166 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:27.185 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:27.186 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2024-05-04T12:26:27.514 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:27.521 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:27.522 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:27.737 INFO:teuthology.orchestra.run.smithi052.stdout:generic/019 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:26:27.743 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/019 2024-05-04T12:26:27.744 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/019 2024-05-04T12:26:27.744 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:27.746 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:27.758 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:27.758 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2024-05-04T12:26:28.170 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:28.177 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:28.177 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:40.869 INFO:teuthology.orchestra.run.smithi052.stdout:generic/020 - output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/020.out.bad) 2024-05-04T12:26:40.888 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/020.out 2024-05-04 12:19:31.956993061 +0000 2024-05-04T12:26:40.888 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/020.out.bad 2024-05-04 12:26:40.860702585 +0000 2024-05-04T12:26:40.888 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -47,9 +47,13 @@ 2024-05-04T12:26:40.888 INFO:teuthology.orchestra.run.smithi052.stdout: user.snrub="fish2\012" 2024-05-04T12:26:40.888 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:40.888 INFO:teuthology.orchestra.run.smithi052.stdout: *** really long value 2024-05-04T12:26:40.888 INFO:teuthology.orchestra.run.smithi052.stdout: -0000000 00 2024-05-04T12:26:40.888 INFO:teuthology.orchestra.run.smithi052.stdout: -* 2024-05-04T12:26:40.888 INFO:teuthology.orchestra.run.smithi052.stdout: -ATTRSIZE 2024-05-04T12:26:40.888 INFO:teuthology.orchestra.run.smithi052.stdout: +attr_set: No space left on device 2024-05-04T12:26:40.889 INFO:teuthology.orchestra.run.smithi052.stdout: +Could not set "long_attr" for 2024-05-04T12:26:40.889 INFO:teuthology.orchestra.run.smithi052.stdout: +attr_get: No data available 2024-05-04T12:26:40.889 INFO:teuthology.orchestra.run.smithi052.stdout: +Could not get "long_attr" for 2024-05-04T12:26:40.889 INFO:teuthology.orchestra.run.smithi052.stdout: +0000000 2024-05-04T12:26:40.889 INFO:teuthology.orchestra.run.smithi052.stdout: +attr_remove: No data available 2024-05-04T12:26:40.889 INFO:teuthology.orchestra.run.smithi052.stdout: +Could not remove "long_attr" for 2024-05-04T12:26:40.889 INFO:teuthology.orchestra.run.smithi052.stdout: *** set/get/remove really long names (expect failure) 2024-05-04T12:26:40.889 INFO:teuthology.orchestra.run.smithi052.stdout: attr_set: Invalid argument 2024-05-04T12:26:40.889 INFO:teuthology.orchestra.run.smithi052.stdout: Could not set "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" for 2024-05-04T12:26:40.891 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/020 2024-05-04T12:26:40.891 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/020 2024-05-04T12:26:40.891 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T12:26:40.893 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:40.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T12:26:40.908 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T12:26:40.908 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T12:26:40.908 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2024-05-04T12:26:41.406 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:41.412 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:41.412 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:43.175 INFO:teuthology.orchestra.run.smithi052.stdout:generic/021 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-04T12:26:43.185 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/021 2024-05-04T12:26:43.185 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/021 2024-05-04T12:26:43.186 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:43.190 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:43.235 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:43.236 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2024-05-04T12:26:43.757 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:43.763 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:43.763 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:45.554 INFO:teuthology.orchestra.run.smithi052.stdout:generic/022 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-04T12:26:45.561 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/022 2024-05-04T12:26:45.561 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/022 2024-05-04T12:26:45.561 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:45.565 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:45.672 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:45.673 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2024-05-04T12:26:46.067 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:46.076 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:46.077 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:48.252 INFO:teuthology.orchestra.run.smithi052.stdout:generic/023 2s 2024-05-04T12:26:48.263 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/023 2024-05-04T12:26:48.263 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:48.267 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:48.283 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:48.283 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2024-05-04T12:26:48.799 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:48.804 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:48.804 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:49.050 INFO:teuthology.orchestra.run.smithi052.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2024-05-04T12:26:49.054 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/024 2024-05-04T12:26:49.054 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/024 2024-05-04T12:26:49.054 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:49.057 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:49.077 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:49.078 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2024-05-04T12:26:49.577 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:49.592 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:49.593 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:49.840 INFO:teuthology.orchestra.run.smithi052.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2024-05-04T12:26:49.844 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/025 2024-05-04T12:26:49.844 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/025 2024-05-04T12:26:49.845 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:49.848 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:49.947 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:49.947 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2024-05-04T12:26:50.360 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:50.374 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:50.375 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:50.621 INFO:teuthology.orchestra.run.smithi052.stdout:generic/026 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-05-04T12:26:50.624 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/026 2024-05-04T12:26:50.624 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/026 2024-05-04T12:26:50.625 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:50.628 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:50.656 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:50.656 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2024-05-04T12:26:51.058 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:51.066 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:51.066 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:51.252 INFO:teuthology.orchestra.run.smithi052.stdout:generic/027 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:26:51.258 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/027 2024-05-04T12:26:51.258 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/027 2024-05-04T12:26:51.258 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:51.261 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:51.273 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:51.273 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2024-05-04T12:26:51.612 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:51.621 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:51.621 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:56.882 INFO:teuthology.orchestra.run.smithi052.stdout:generic/028 5s 2024-05-04T12:26:56.958 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/028 2024-05-04T12:26:56.958 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:56.965 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:56.982 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:56.982 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2024-05-04T12:26:57.301 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:57.316 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:57.316 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:57.497 INFO:teuthology.orchestra.run.smithi052.stdout:generic/029 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:26:57.504 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/029 2024-05-04T12:26:57.504 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/029 2024-05-04T12:26:57.504 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:57.507 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:57.524 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:57.524 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2024-05-04T12:26:57.894 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:57.907 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:57.907 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:58.093 INFO:teuthology.orchestra.run.smithi052.stdout:generic/030 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:26:58.097 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/030 2024-05-04T12:26:58.097 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/030 2024-05-04T12:26:58.098 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:58.103 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:58.114 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:58.114 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2024-05-04T12:26:58.582 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:58.605 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:58.605 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:58.790 INFO:teuthology.orchestra.run.smithi052.stdout:generic/031 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:26:58.798 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/031 2024-05-04T12:26:58.798 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/031 2024-05-04T12:26:58.798 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:58.815 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:58.829 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:58.830 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2024-05-04T12:26:59.227 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:59.233 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:59.233 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:59.415 INFO:teuthology.orchestra.run.smithi052.stdout:generic/032 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:26:59.422 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/032 2024-05-04T12:26:59.422 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/032 2024-05-04T12:26:59.423 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:59.425 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:59.435 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:59.436 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2024-05-04T12:26:59.742 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:26:59.748 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:26:59.748 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:59.926 INFO:teuthology.orchestra.run.smithi052.stdout:generic/033 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:26:59.934 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/033 2024-05-04T12:26:59.934 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/033 2024-05-04T12:26:59.934 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:26:59.939 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:26:59.964 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:26:59.964 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2024-05-04T12:27:00.354 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:00.372 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:00.581 INFO:teuthology.orchestra.run.smithi052.stdout:generic/034 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:00.589 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/034 2024-05-04T12:27:00.589 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/034 2024-05-04T12:27:00.590 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:00.617 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:00.617 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2024-05-04T12:27:00.989 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:01.005 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:01.315 INFO:teuthology.orchestra.run.smithi052.stdout:generic/035 0s 2024-05-04T12:27:01.323 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/035 2024-05-04T12:27:01.323 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:01.327 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:01.397 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:01.398 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2024-05-04T12:27:01.833 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:01.840 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:01.840 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:12.257 INFO:teuthology.orchestra.run.smithi052.stdout:generic/036 11s 2024-05-04T12:27:12.265 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/036 2024-05-04T12:27:12.265 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:12.351 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:12.351 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2024-05-04T12:27:12.779 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:12.802 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:12.954 INFO:teuthology.orchestra.run.smithi052.stdout:generic/037 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:12.967 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/037 2024-05-04T12:27:12.967 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/037 2024-05-04T12:27:12.967 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:12.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:12.995 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2024-05-04T12:27:13.327 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:13.335 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:13.513 INFO:teuthology.orchestra.run.smithi052.stdout:generic/038 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:13.520 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/038 2024-05-04T12:27:13.520 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/038 2024-05-04T12:27:13.520 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:13.534 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:13.535 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2024-05-04T12:27:13.907 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:13.912 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:13.913 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:14.120 INFO:teuthology.orchestra.run.smithi052.stdout:generic/039 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:14.125 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/039 2024-05-04T12:27:14.125 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/039 2024-05-04T12:27:14.125 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:14.128 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:14.151 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:14.151 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2024-05-04T12:27:14.544 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:14.556 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:14.556 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:14.762 INFO:teuthology.orchestra.run.smithi052.stdout:generic/040 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:14.769 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/040 2024-05-04T12:27:14.769 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/040 2024-05-04T12:27:14.769 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:14.791 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:14.805 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:14.805 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2024-05-04T12:27:15.163 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:15.168 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:15.168 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:15.384 INFO:teuthology.orchestra.run.smithi052.stdout:generic/041 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:15.396 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/041 2024-05-04T12:27:15.396 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/041 2024-05-04T12:27:15.396 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:15.403 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:15.422 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:15.422 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2024-05-04T12:27:15.762 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:15.769 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:15.769 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:15.949 INFO:teuthology.orchestra.run.smithi052.stdout:generic/042 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:15.958 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/042 2024-05-04T12:27:15.958 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/042 2024-05-04T12:27:15.958 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:15.970 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:15.978 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:15.978 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2024-05-04T12:27:16.376 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:16.382 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:16.382 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:16.588 INFO:teuthology.orchestra.run.smithi052.stdout:generic/043 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:16.592 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/043 2024-05-04T12:27:16.592 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/043 2024-05-04T12:27:16.592 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:16.602 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:16.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:16.615 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2024-05-04T12:27:17.012 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:17.016 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:17.016 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:17.192 INFO:teuthology.orchestra.run.smithi052.stdout:generic/044 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:17.197 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/044 2024-05-04T12:27:17.197 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/044 2024-05-04T12:27:17.197 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:17.202 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:17.238 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:17.238 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2024-05-04T12:27:17.557 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:17.561 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:17.561 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:17.752 INFO:teuthology.orchestra.run.smithi052.stdout:generic/045 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:17.757 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/045 2024-05-04T12:27:17.757 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/045 2024-05-04T12:27:17.757 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:17.760 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:17.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:17.774 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2024-05-04T12:27:18.114 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:18.120 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:18.120 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:18.283 INFO:teuthology.orchestra.run.smithi052.stdout:generic/046 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:18.292 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/046 2024-05-04T12:27:18.292 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/046 2024-05-04T12:27:18.293 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:18.295 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:18.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:18.309 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2024-05-04T12:27:18.625 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:18.636 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:18.637 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:18.809 INFO:teuthology.orchestra.run.smithi052.stdout:generic/047 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:18.813 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/047 2024-05-04T12:27:18.813 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/047 2024-05-04T12:27:18.813 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:18.816 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:18.842 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:18.842 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2024-05-04T12:27:19.205 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:19.237 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:19.237 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:19.392 INFO:teuthology.orchestra.run.smithi052.stdout:generic/048 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:19.398 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/048 2024-05-04T12:27:19.399 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/048 2024-05-04T12:27:19.399 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:19.401 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:19.416 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:19.416 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2024-05-04T12:27:19.783 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:19.793 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:19.793 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:19.984 INFO:teuthology.orchestra.run.smithi052.stdout:generic/049 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/049 2024-05-04T12:27:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/049 2024-05-04T12:27:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:20.008 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:20.027 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:20.027 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2024-05-04T12:27:20.444 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:20.449 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:20.450 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:20.610 INFO:teuthology.orchestra.run.smithi052.stdout:generic/050 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:20.615 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/050 2024-05-04T12:27:20.615 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/050 2024-05-04T12:27:20.615 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:20.622 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:20.645 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:20.645 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2024-05-04T12:27:21.001 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:21.006 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:21.007 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:21.190 INFO:teuthology.orchestra.run.smithi052.stdout:generic/051 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:21.195 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/051 2024-05-04T12:27:21.195 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/051 2024-05-04T12:27:21.195 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:21.198 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:21.209 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:21.210 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2024-05-04T12:27:21.553 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:21.560 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:21.560 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:21.760 INFO:teuthology.orchestra.run.smithi052.stdout:generic/052 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:21.766 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/052 2024-05-04T12:27:21.766 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/052 2024-05-04T12:27:21.766 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:21.769 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:21.778 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:21.778 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2024-05-04T12:27:22.113 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:22.118 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:22.118 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:22.296 INFO:teuthology.orchestra.run.smithi052.stdout:generic/053 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:22.303 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/053 2024-05-04T12:27:22.304 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/053 2024-05-04T12:27:22.304 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:22.306 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:22.321 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:22.321 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2024-05-04T12:27:22.673 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:22.685 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:22.686 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:22.870 INFO:teuthology.orchestra.run.smithi052.stdout:generic/054 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:22.876 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/054 2024-05-04T12:27:22.876 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/054 2024-05-04T12:27:22.876 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:22.887 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:22.931 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:22.932 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2024-05-04T12:27:23.290 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:23.296 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:23.296 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:23.492 INFO:teuthology.orchestra.run.smithi052.stdout:generic/055 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:23.496 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/055 2024-05-04T12:27:23.497 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/055 2024-05-04T12:27:23.497 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:23.500 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:23.516 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:23.516 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2024-05-04T12:27:23.874 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:23.897 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:23.897 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:24.102 INFO:teuthology.orchestra.run.smithi052.stdout:generic/056 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:24.106 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/056 2024-05-04T12:27:24.106 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/056 2024-05-04T12:27:24.106 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:24.109 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:24.123 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:24.125 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2024-05-04T12:27:24.514 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:24.520 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:24.520 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:24.730 INFO:teuthology.orchestra.run.smithi052.stdout:generic/057 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:24.742 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/057 2024-05-04T12:27:24.742 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/057 2024-05-04T12:27:24.742 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:24.745 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:24.758 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:24.758 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2024-05-04T12:27:25.115 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:25.120 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:25.120 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:26.789 INFO:teuthology.orchestra.run.smithi052.stdout:generic/058 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-04T12:27:26.794 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/058 2024-05-04T12:27:26.794 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/058 2024-05-04T12:27:26.794 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:26.797 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:26.908 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:26.908 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2024-05-04T12:27:27.500 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:27.509 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:27.509 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:27.692 INFO:teuthology.orchestra.run.smithi052.stdout:generic/059 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:27.700 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/059 2024-05-04T12:27:27.700 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/059 2024-05-04T12:27:27.700 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:27.706 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:27.723 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:27.723 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2024-05-04T12:27:28.095 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:28.100 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:28.100 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:29.877 INFO:teuthology.orchestra.run.smithi052.stdout:generic/060 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-04T12:27:29.885 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/060 2024-05-04T12:27:29.885 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/060 2024-05-04T12:27:29.885 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:29.891 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:29.909 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:29.910 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2024-05-04T12:27:30.518 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:30.524 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:30.524 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:32.274 INFO:teuthology.orchestra.run.smithi052.stdout:generic/061 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-04T12:27:32.278 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/061 2024-05-04T12:27:32.278 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/061 2024-05-04T12:27:32.278 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:32.280 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:32.302 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:32.303 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2024-05-04T12:27:32.882 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:32.888 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:32.889 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:33.112 INFO:teuthology.orchestra.run.smithi052.stdout:generic/062 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:33.116 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/062 2024-05-04T12:27:33.116 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/062 2024-05-04T12:27:33.117 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:33.120 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:33.132 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:33.132 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2024-05-04T12:27:33.473 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:33.481 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:33.482 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:35.294 INFO:teuthology.orchestra.run.smithi052.stdout:generic/063 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-04T12:27:35.305 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/063 2024-05-04T12:27:35.305 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/063 2024-05-04T12:27:35.305 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:35.321 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:35.322 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2024-05-04T12:27:35.929 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:35.934 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:35.934 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:36.136 INFO:teuthology.orchestra.run.smithi052.stdout:generic/064 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:36.139 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/064 2024-05-04T12:27:36.140 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/064 2024-05-04T12:27:36.140 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:36.142 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:36.156 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:36.156 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2024-05-04T12:27:36.517 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:36.524 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:36.524 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:36.713 INFO:teuthology.orchestra.run.smithi052.stdout:generic/065 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:36.718 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/065 2024-05-04T12:27:36.718 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/065 2024-05-04T12:27:36.718 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:36.722 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:36.734 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:36.734 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2024-05-04T12:27:37.137 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:37.142 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:37.142 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:37.355 INFO:teuthology.orchestra.run.smithi052.stdout:generic/066 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:37.360 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/066 2024-05-04T12:27:37.360 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/066 2024-05-04T12:27:37.360 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:37.363 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:37.384 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:37.384 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2024-05-04T12:27:37.680 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:37.688 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:37.688 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:37.930 INFO:teuthology.orchestra.run.smithi052.stdout:generic/067 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:37.935 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/067 2024-05-04T12:27:37.935 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/067 2024-05-04T12:27:37.935 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:37.941 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:37.966 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:37.967 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2024-05-04T12:27:38.294 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:38.316 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:38.316 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:38.530 INFO:teuthology.orchestra.run.smithi052.stdout:generic/068 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:38.534 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/068 2024-05-04T12:27:38.534 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/068 2024-05-04T12:27:38.534 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:38.538 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:38.563 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:38.563 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2024-05-04T12:27:38.957 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:38.966 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:38.966 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:39.145 INFO:teuthology.orchestra.run.smithi052.stdout:generic/069 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:27:39.152 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/069 2024-05-04T12:27:39.152 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/069 2024-05-04T12:27:39.152 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:27:39.157 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:27:39.169 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:27:39.169 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2024-05-04T12:27:39.503 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:27:39.514 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:27:39.514 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:29:15.517 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:29:15.519 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:29:21.789 INFO:teuthology.orchestra.run.smithi052.stdout:generic/070 102s 2024-05-04T12:29:21.796 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/070 2024-05-04T12:29:21.796 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:29:21.804 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:29:21.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:29:21.973 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2024-05-04T12:29:22.349 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:29:22.354 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:29:22.354 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:29:22.516 INFO:teuthology.orchestra.run.smithi052.stdout:generic/071 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:29:22.521 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/071 2024-05-04T12:29:22.521 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/071 2024-05-04T12:29:22.521 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:29:22.523 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:29:22.546 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:29:22.546 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2024-05-04T12:29:22.780 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:29:22.784 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:29:22.784 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:29:24.191 INFO:teuthology.orchestra.run.smithi052.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-05-04T12:29:24.195 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/072 2024-05-04T12:29:24.195 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/072 2024-05-04T12:29:24.195 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:29:24.201 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:29:24.217 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:29:24.218 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2024-05-04T12:29:24.487 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:29:24.492 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:29:24.492 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:29:24.667 INFO:teuthology.orchestra.run.smithi052.stdout:generic/073 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T12:29:24.672 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/073 2024-05-04T12:29:24.672 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/073 2024-05-04T12:29:24.672 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:29:24.678 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:29:24.692 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:29:24.692 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2024-05-04T12:29:24.951 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:29:24.961 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:29:24.962 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:29:45.794 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:29:45.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:30:15.871 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:30:15.873 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:30:45.960 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:30:45.962 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:31:16.043 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:31:16.046 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:31:46.481 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:31:46.483 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:32:16.720 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:32:16.722 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:32:46.761 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:32:46.763 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:33:16.817 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:33:16.819 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:33:46.929 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:33:46.932 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:34:17.144 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:34:17.146 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:34:47.288 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:34:47.322 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:35:17.527 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:35:17.530 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:35:47.614 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:35:47.617 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:36:17.720 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:36:17.722 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:36:47.776 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:36:47.779 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:37:18.132 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:37:18.135 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:37:48.396 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:37:48.399 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:38:18.851 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:38:18.854 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:38:49.278 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:38:49.280 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:39:19.344 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:39:19.346 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:39:49.478 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:39:49.480 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:40:19.625 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:40:19.627 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:40:49.673 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:40:49.676 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:41:19.754 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:41:19.756 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:41:49.952 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:41:49.954 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:42:20.229 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:42:20.231 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:42:50.314 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:42:50.316 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:43:20.591 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:43:20.594 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:43:50.675 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:43:50.677 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:44:20.731 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:44:20.734 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:44:50.796 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:44:50.798 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:45:20.852 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:45:20.856 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:45:50.901 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:45:50.904 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:46:20.947 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:46:20.949 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:46:51.363 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:46:51.366 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:47:21.414 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:47:21.417 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:47:51.711 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:47:51.714 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:48:21.960 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:48:21.963 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:48:52.010 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:48:52.012 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:49:22.117 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:49:22.119 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:49:52.339 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:49:52.341 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:50:22.394 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:50:22.396 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:50:52.500 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:50:52.503 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:51:22.816 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:51:22.818 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:51:52.849 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:51:52.852 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:52:22.961 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:52:22.963 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:52:53.050 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:52:53.053 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:52:53.097 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T12:52:53.096+0000 7fb74db8d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34455) UID: 0 2024-05-04T12:52:53.097 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T12:52:53.096+0000 7f1412d90640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34455) UID: 0 2024-05-04T12:52:53.098 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T12:52:53.096+0000 7f81dc32b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34455) UID: 0 2024-05-04T12:52:53.098 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T12:52:53.096+0000 7fc53dde6640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34455) UID: 0 2024-05-04T12:56:06.592 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:56:06.595 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:56:37.002 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:56:37.004 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:57:07.059 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:57:07.062 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:57:37.115 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:57:37.117 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:58:07.220 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:58:07.223 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:58:37.317 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:58:37.320 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:59:05.686 INFO:teuthology.orchestra.run.smithi052.stdout:generic/074 1781s 2024-05-04T12:59:05.781 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/074 2024-05-04T12:59:05.781 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T12:59:05.786 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:59:05.800 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T12:59:05.800 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2024-05-04T12:59:06.670 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T12:59:06.674 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T12:59:06.674 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T12:59:07.709 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:59:07.712 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:59:37.756 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T12:59:37.759 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:00:08.004 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:00:08.007 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:00:34.517 INFO:teuthology.orchestra.run.smithi052.stdout:generic/075 88s 2024-05-04T13:00:34.524 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/075 2024-05-04T13:00:34.524 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:00:34.528 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:34.757 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:00:34.757 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2024-05-04T13:00:35.096 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:00:35.100 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:00:35.100 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:35.221 INFO:teuthology.orchestra.run.smithi052.stdout:generic/076 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:00:35.225 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/076 2024-05-04T13:00:35.225 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/076 2024-05-04T13:00:35.225 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:00:35.228 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:35.237 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:00:35.238 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2024-05-04T13:00:35.411 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:00:35.415 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:00:35.415 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:36.475 INFO:teuthology.orchestra.run.smithi052.stdout:generic/077 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:00:36.481 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/077 2024-05-04T13:00:36.481 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/077 2024-05-04T13:00:36.481 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:00:36.485 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:36.496 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:00:36.497 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2024-05-04T13:00:36.670 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:00:36.674 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:00:36.674 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:36.818 INFO:teuthology.orchestra.run.smithi052.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2024-05-04T13:00:36.822 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/078 2024-05-04T13:00:36.822 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/078 2024-05-04T13:00:36.822 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:00:36.824 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:36.833 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:00:36.833 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2024-05-04T13:00:37.324 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:00:37.328 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:00:37.328 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:37.498 INFO:teuthology.orchestra.run.smithi052.stdout:generic/079 [not run] file system doesn't support chattr +ia 2024-05-04T13:00:37.502 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/079 2024-05-04T13:00:37.502 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/079 2024-05-04T13:00:37.502 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:00:37.504 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:37.513 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:00:37.513 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2024-05-04T13:00:37.665 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:00:37.669 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:00:37.669 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:38.068 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:00:38.070 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:00:39.866 INFO:teuthology.orchestra.run.smithi052.stdout:generic/080 2s 2024-05-04T13:00:39.871 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/080 2024-05-04T13:00:39.871 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:00:39.874 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:39.885 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:00:39.886 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2024-05-04T13:00:40.084 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:00:40.089 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:00:40.089 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:40.217 INFO:teuthology.orchestra.run.smithi052.stdout:generic/081 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:00:40.221 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/081 2024-05-04T13:00:40.221 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/081 2024-05-04T13:00:40.221 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:00:40.224 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:40.234 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:00:40.234 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2024-05-04T13:00:40.406 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:00:40.411 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:00:40.411 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:40.540 INFO:teuthology.orchestra.run.smithi052.stdout:generic/082 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:00:40.544 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/082 2024-05-04T13:00:40.544 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/082 2024-05-04T13:00:40.544 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:00:40.547 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:40.557 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:00:40.557 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2024-05-04T13:00:40.728 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:00:40.733 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:00:40.733 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:40.841 INFO:teuthology.orchestra.run.smithi052.stdout:generic/083 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:00:40.845 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/083 2024-05-04T13:00:40.845 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/083 2024-05-04T13:00:40.845 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:00:40.847 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:40.855 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:00:40.855 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2024-05-04T13:00:41.002 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:00:41.006 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:00:41.006 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:41.107 INFO:teuthology.orchestra.run.smithi052.stdout:generic/084 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:00:41.111 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/084 2024-05-04T13:00:41.111 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/084 2024-05-04T13:00:41.111 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:00:41.113 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:41.121 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:00:41.121 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2024-05-04T13:00:41.269 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:00:41.273 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:00:41.274 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:41.417 INFO:teuthology.orchestra.run.smithi052.stdout:generic/085 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:00:41.422 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/085 2024-05-04T13:00:41.422 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/085 2024-05-04T13:00:41.422 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:00:41.424 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:41.432 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:00:41.432 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2024-05-04T13:00:41.583 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:00:41.587 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:00:41.588 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:47.013 INFO:teuthology.orchestra.run.smithi052.stdout:generic/086 5s 2024-05-04T13:00:47.124 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/086 2024-05-04T13:00:47.124 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:00:47.130 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:47.214 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:00:47.214 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2024-05-04T13:00:48.843 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:00:48.848 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:00:48.848 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:49.244 INFO:teuthology.orchestra.run.smithi052.stdout:generic/087 1s 2024-05-04T13:00:49.249 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/087 2024-05-04T13:00:49.249 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:00:49.251 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:49.259 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:00:49.259 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2024-05-04T13:00:49.766 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:00:49.770 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:00:49.770 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:49.925 INFO:teuthology.orchestra.run.smithi052.stdout:generic/088 0s 2024-05-04T13:00:49.930 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/088 2024-05-04T13:00:49.930 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:00:49.933 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:00:49.941 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:00:49.941 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2024-05-04T13:00:50.460 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:00:50.464 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:00:50.464 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:01:08.132 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:01:08.134 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:01:38.253 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:01:38.255 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:02:08.305 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:02:08.307 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:02:38.425 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:02:38.427 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:03:08.734 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:03:08.737 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:03:39.041 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:03:39.044 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:04:09.083 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:04:09.085 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:04:09.122 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-04T13:04:09.122+0000 7f6f26d7f640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 123025) UID: 0 2024-05-04T13:04:09.122 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T13:04:09.122+0000 7fbf91e1b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 123025) UID: 0 2024-05-04T13:07:17.557 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:07:17.560 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:07:47.632 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:07:47.634 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:07:47.650 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T13:07:47.651+0000 7f3c99d9c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123252) UID: 0 2024-05-04T13:07:47.650 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T13:07:47.651+0000 7f7b9124a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123252) UID: 0 2024-05-04T13:07:47.651 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T13:07:47.651+0000 7fd3bf2df640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123252) UID: 0 2024-05-04T13:07:47.651 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T13:07:47.651+0000 7f17989d7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123252) UID: 0 2024-05-04T13:12:00.903 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:12:00.906 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:12:00.927 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T13:12:00.930+0000 7f1412d90640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34562) UID: 0 2024-05-04T13:12:00.928 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T13:12:00.930+0000 7fb74db8d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34562) UID: 0 2024-05-04T13:12:00.928 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T13:12:00.930+0000 7f81dc32b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34562) UID: 0 2024-05-04T13:12:00.928 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T13:12:00.930+0000 7fc53dde6640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34562) UID: 0 2024-05-04T13:12:01.170 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-04T13:12:01.172+0000 7f6f26d7f640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 125275) UID: 0 2024-05-04T13:12:01.170 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T13:12:01.172+0000 7fbf91e1b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 125275) UID: 0 2024-05-04T13:17:15.718 INFO:teuthology.orchestra.run.smithi052.stdout:generic/089 985s 2024-05-04T13:17:15.849 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/089 2024-05-04T13:17:15.850 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:17:15.854 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:17:15.871 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:17:15.871 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/090) 2024-05-04T13:17:17.664 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:17:17.669 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:17:17.669 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:17:18.320 INFO:teuthology.orchestra.run.smithi052.stdout:generic/090 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:17:18.327 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/090 2024-05-04T13:17:18.327 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/090 2024-05-04T13:17:18.328 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:17:18.331 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:17:18.344 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:17:18.344 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/091) 2024-05-04T13:17:18.561 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:17:18.566 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:17:18.566 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:18:50.744 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:18:50.748 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:18:50.819 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-04T13:18:50.819+0000 7f6f26d7f640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 126176) UID: 0 2024-05-04T13:18:50.819 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T13:18:50.819+0000 7fbf91e1b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 126176) UID: 0 2024-05-04T13:21:25.562 INFO:teuthology.orchestra.run.smithi052.stdout:generic/091 247s 2024-05-04T13:21:25.570 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/091 2024-05-04T13:21:25.570 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:21:25.574 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:25.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:21:25.810 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/092) 2024-05-04T13:21:26.155 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:21:26.159 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:21:26.159 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:26.343 INFO:teuthology.orchestra.run.smithi052.stdout:generic/092 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-04T13:21:26.346 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/092 2024-05-04T13:21:26.346 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/092 2024-05-04T13:21:26.347 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:21:26.349 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:26.357 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:21:26.357 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/093) 2024-05-04T13:21:26.853 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:21:26.857 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:21:26.857 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:28.047 INFO:teuthology.orchestra.run.smithi052.stdout:generic/093 2s 2024-05-04T13:21:28.053 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/093 2024-05-04T13:21:28.053 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:21:28.056 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:28.065 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:21:28.065 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/094) 2024-05-04T13:21:28.554 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:21:28.558 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:21:28.558 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:28.659 INFO:teuthology.orchestra.run.smithi052.stdout:generic/094 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:21:28.663 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/094 2024-05-04T13:21:28.663 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/094 2024-05-04T13:21:28.663 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:21:28.665 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:28.673 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:21:28.673 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/095) 2024-05-04T13:21:28.824 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:21:28.828 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:21:28.828 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:28.931 INFO:teuthology.orchestra.run.smithi052.stdout:generic/095 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:21:28.934 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/095 2024-05-04T13:21:28.934 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/095 2024-05-04T13:21:28.934 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:21:28.936 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:28.943 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:21:28.944 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/096) 2024-05-04T13:21:29.091 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:21:29.094 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:21:29.094 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:generic/096 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:21:29.217 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/096 2024-05-04T13:21:29.217 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/096 2024-05-04T13:21:29.217 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:21:29.219 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:29.228 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:21:29.228 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/097) 2024-05-04T13:21:29.379 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:21:29.383 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:21:29.383 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:30.015 INFO:teuthology.orchestra.run.smithi052.stdout:generic/097 1s 2024-05-04T13:21:30.020 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/097 2024-05-04T13:21:30.020 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:21:30.023 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:30.030 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:21:30.030 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/098) 2024-05-04T13:21:30.178 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:21:30.182 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:21:30.183 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:30.289 INFO:teuthology.orchestra.run.smithi052.stdout:generic/098 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:21:30.293 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/098 2024-05-04T13:21:30.293 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/098 2024-05-04T13:21:30.293 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:21:30.295 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:30.303 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:21:30.303 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-05-04T13:21:30.452 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:21:30.456 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:21:30.457 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:30.606 INFO:teuthology.orchestra.run.smithi052.stdout:generic/099 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-05-04T13:21:30.610 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/099 2024-05-04T13:21:30.610 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/099 2024-05-04T13:21:30.610 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:21:30.612 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:30.620 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:21:30.620 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/100) 2024-05-04T13:21:30.771 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:21:30.774 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:21:30.775 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:21:47.484 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:21:47.487 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:21:47.508 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T13:21:47.510+0000 7fb74db8d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34592) UID: 0 2024-05-04T13:21:47.508 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T13:21:47.510+0000 7f81dc32b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34592) UID: 0 2024-05-04T13:21:47.509 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T13:21:47.510+0000 7f1412d90640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34592) UID: 0 2024-05-04T13:21:47.509 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T13:21:47.510+0000 7fc53dde6640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34592) UID: 0 2024-05-04T13:22:10.957 INFO:teuthology.orchestra.run.smithi052.stdout:generic/100 40s 2024-05-04T13:22:11.010 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/100 2024-05-04T13:22:11.010 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:22:11.014 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:11.035 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:22:11.035 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/101) 2024-05-04T13:22:11.225 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:22:11.229 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:22:11.229 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:12.328 INFO:teuthology.orchestra.run.smithi052.stdout:generic/101 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:22:12.334 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/101 2024-05-04T13:22:12.334 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/101 2024-05-04T13:22:12.335 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:22:12.339 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:12.352 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:22:12.352 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/102) 2024-05-04T13:22:12.544 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:22:12.548 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:22:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:12.652 INFO:teuthology.orchestra.run.smithi052.stdout:generic/102 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:22:12.656 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/102 2024-05-04T13:22:12.656 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/102 2024-05-04T13:22:12.656 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:22:12.659 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:12.666 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:22:12.667 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/103) 2024-05-04T13:22:12.818 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:22:12.821 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:22:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:12.922 INFO:teuthology.orchestra.run.smithi052.stdout:generic/103 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:22:12.925 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/103 2024-05-04T13:22:12.926 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/103 2024-05-04T13:22:12.926 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:22:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:12.935 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:22:12.935 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/104) 2024-05-04T13:22:13.087 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:22:13.091 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:22:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:13.205 INFO:teuthology.orchestra.run.smithi052.stdout:generic/104 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:22:13.209 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/104 2024-05-04T13:22:13.209 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/104 2024-05-04T13:22:13.209 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:22:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:13.219 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:22:13.219 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/105) 2024-05-04T13:22:13.374 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:22:13.378 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:22:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:13.495 INFO:teuthology.orchestra.run.smithi052.stdout:generic/105 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:22:13.498 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/105 2024-05-04T13:22:13.499 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/105 2024-05-04T13:22:13.499 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:22:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:13.509 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:22:13.509 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/106) 2024-05-04T13:22:13.662 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:22:13.666 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:22:13.666 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:13.789 INFO:teuthology.orchestra.run.smithi052.stdout:generic/106 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:22:13.793 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/106 2024-05-04T13:22:13.793 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/106 2024-05-04T13:22:13.793 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:22:13.795 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:13.803 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:22:13.803 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/107) 2024-05-04T13:22:13.959 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:22:13.963 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:22:13.963 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:14.089 INFO:teuthology.orchestra.run.smithi052.stdout:generic/107 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:22:14.093 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/107 2024-05-04T13:22:14.093 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/107 2024-05-04T13:22:14.093 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:22:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:14.102 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:22:14.103 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/108) 2024-05-04T13:22:14.254 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:22:14.258 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:22:14.258 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:14.585 INFO:teuthology.orchestra.run.smithi052.stdout:generic/108 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:22:14.593 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/108 2024-05-04T13:22:14.593 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/108 2024-05-04T13:22:14.593 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:22:14.598 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:14.610 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:22:14.610 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/109) 2024-05-04T13:22:14.813 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:22:14.817 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:22:14.818 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:14.932 INFO:teuthology.orchestra.run.smithi052.stdout:generic/109 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:22:14.936 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/109 2024-05-04T13:22:14.936 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/109 2024-05-04T13:22:14.936 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:22:14.939 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:14.948 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:22:14.948 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/110) 2024-05-04T13:22:15.107 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:22:15.111 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:22:15.111 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:15.278 INFO:teuthology.orchestra.run.smithi052.stdout:generic/110 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T13:22:15.282 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/110 2024-05-04T13:22:15.282 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/110 2024-05-04T13:22:15.282 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:22:15.284 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:15.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:22:15.292 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/111) 2024-05-04T13:22:15.799 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:22:15.804 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:22:15.804 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:15.956 INFO:teuthology.orchestra.run.smithi052.stdout:generic/111 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T13:22:15.959 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/111 2024-05-04T13:22:15.959 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/111 2024-05-04T13:22:15.960 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:22:15.961 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:22:15.969 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:22:15.970 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/112) 2024-05-04T13:22:16.445 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:22:16.449 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:22:16.449 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:23:50.936 INFO:teuthology.orchestra.run.smithi052.stdout:generic/112 94s 2024-05-04T13:23:50.944 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/112 2024-05-04T13:23:50.944 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:23:50.948 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:23:50.961 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:23:50.961 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/113) 2024-05-04T13:23:51.500 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:23:51.504 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:23:51.504 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:08.672 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:24:08.675 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:24:33.830 INFO:teuthology.orchestra.run.smithi052.stdout:generic/113 42s 2024-05-04T13:24:33.836 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/113 2024-05-04T13:24:33.836 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:24:33.839 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:34.042 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:24:34.042 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/114) 2024-05-04T13:24:34.427 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:24:34.431 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:24:34.431 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:34.586 INFO:teuthology.orchestra.run.smithi052.stdout:generic/114 [not run] Sparse files are not supported or do not work as expected 2024-05-04T13:24:34.589 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/114 2024-05-04T13:24:34.589 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/114 2024-05-04T13:24:34.589 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:24:34.592 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:34.600 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:24:34.600 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/115) 2024-05-04T13:24:34.744 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:24:34.748 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:24:34.748 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:34.895 INFO:teuthology.orchestra.run.smithi052.stdout:generic/115 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T13:24:34.899 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/115 2024-05-04T13:24:34.899 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/115 2024-05-04T13:24:34.899 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:24:34.901 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:34.909 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:24:34.909 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/116) 2024-05-04T13:24:35.421 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:24:35.425 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:24:35.425 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:35.568 INFO:teuthology.orchestra.run.smithi052.stdout:generic/116 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T13:24:35.572 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/116 2024-05-04T13:24:35.572 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/116 2024-05-04T13:24:35.572 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:24:35.575 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:35.583 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:24:35.583 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/117) 2024-05-04T13:24:36.089 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:24:36.093 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:24:36.093 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:36.196 INFO:teuthology.orchestra.run.smithi052.stdout:generic/117 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:24:36.200 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/117 2024-05-04T13:24:36.200 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/117 2024-05-04T13:24:36.200 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:24:36.202 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:36.210 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:24:36.210 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/118) 2024-05-04T13:24:36.360 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:24:36.364 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:24:36.364 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:36.527 INFO:teuthology.orchestra.run.smithi052.stdout:generic/118 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T13:24:36.531 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/118 2024-05-04T13:24:36.531 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/118 2024-05-04T13:24:36.531 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:24:36.533 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:36.541 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:24:36.542 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/119) 2024-05-04T13:24:37.053 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:24:37.058 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:24:37.058 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:37.205 INFO:teuthology.orchestra.run.smithi052.stdout:generic/119 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T13:24:37.209 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/119 2024-05-04T13:24:37.209 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/119 2024-05-04T13:24:37.209 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:24:37.211 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:37.220 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:24:37.220 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/120) 2024-05-04T13:24:37.711 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:24:37.715 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:24:37.715 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:37.815 INFO:teuthology.orchestra.run.smithi052.stdout:generic/120 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:24:37.818 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/120 2024-05-04T13:24:37.818 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/120 2024-05-04T13:24:37.819 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:24:37.821 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:37.829 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:24:37.829 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/121) 2024-05-04T13:24:37.975 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:24:37.979 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:24:37.979 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:38.129 INFO:teuthology.orchestra.run.smithi052.stdout:generic/121 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-05-04T13:24:38.133 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/121 2024-05-04T13:24:38.133 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/121 2024-05-04T13:24:38.133 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:24:38.135 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:38.143 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:24:38.144 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/122) 2024-05-04T13:24:38.640 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:24:38.644 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:24:38.644 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:38.796 INFO:teuthology.orchestra.run.smithi052.stdout:generic/122 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-05-04T13:24:38.800 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/122 2024-05-04T13:24:38.800 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/122 2024-05-04T13:24:38.800 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:24:38.802 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:38.811 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:24:38.811 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/123) 2024-05-04T13:24:38.826 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:24:38.858 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:24:39.303 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:24:39.307 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:24:39.307 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:39.664 INFO:teuthology.orchestra.run.smithi052.stdout:generic/123 0s 2024-05-04T13:24:39.669 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/123 2024-05-04T13:24:39.669 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:24:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:39.908 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:24:39.908 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/124) 2024-05-04T13:24:40.177 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:24:40.181 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:24:40.181 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:40.286 INFO:teuthology.orchestra.run.smithi052.stdout:generic/124 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:24:40.289 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/124 2024-05-04T13:24:40.289 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/124 2024-05-04T13:24:40.289 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:24:40.292 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:24:40.300 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:24:40.300 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/125) 2024-05-04T13:24:40.443 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:24:40.446 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:24:40.446 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:25:08.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:25:08.970 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:25:39.015 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:25:39.018 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:25:41.906 INFO:teuthology.orchestra.run.smithi052.stdout:generic/125 61s 2024-05-04T13:25:41.914 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/125 2024-05-04T13:25:41.914 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:25:41.918 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:25:41.934 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:25:41.934 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/126) 2024-05-04T13:25:42.459 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:25:42.463 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:25:42.463 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:25:42.918 INFO:teuthology.orchestra.run.smithi052.stdout:generic/126 - output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/126.out.bad) 2024-05-04T13:25:42.920 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/126.out 2024-05-04 12:19:31.964992941 +0000 2024-05-04T13:25:42.920 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/126.out.bad 2024-05-04 13:25:42.914583781 +0000 2024-05-04T13:25:42.920 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -1,7 +1,7 @@ 2024-05-04T13:25:42.920 INFO:teuthology.orchestra.run.smithi052.stdout: QA output created by 126 2024-05-04T13:25:42.920 INFO:teuthology.orchestra.run.smithi052.stdout: -x a 001 file owned by (99/99) as user/group(12/100) PASS 2024-05-04T13:25:42.920 INFO:teuthology.orchestra.run.smithi052.stdout: -x a 010 file owned by (99/99) as user/group(200/99) PASS 2024-05-04T13:25:42.920 INFO:teuthology.orchestra.run.smithi052.stdout: -x a 100 file owned by (99/99) as user/group(99/500) PASS 2024-05-04T13:25:42.920 INFO:teuthology.orchestra.run.smithi052.stdout: +x a 001 file owned by (99/99) as user/group(12/100) FAIL 2024-05-04T13:25:42.920 INFO:teuthology.orchestra.run.smithi052.stdout: +x a 010 file owned by (99/99) as user/group(200/99) FAIL 2024-05-04T13:25:42.920 INFO:teuthology.orchestra.run.smithi052.stdout: +x a 100 file owned by (99/99) as user/group(99/500) FAIL 2024-05-04T13:25:42.920 INFO:teuthology.orchestra.run.smithi052.stdout: w a 002 file owned by (99/99) as user/group(12/100) PASS 2024-05-04T13:25:42.920 INFO:teuthology.orchestra.run.smithi052.stdout: w a 020 file owned by (99/99) as user/group(200/99) PASS 2024-05-04T13:25:42.921 INFO:teuthology.orchestra.run.smithi052.stdout: w a 200 file owned by (99/99) as user/group(99/500) PASS 2024-05-04T13:25:42.922 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/126 2024-05-04T13:25:42.922 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/126 2024-05-04T13:25:42.922 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T13:25:42.924 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:25:42.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T13:25:42.933 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T13:25:42.933 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T13:25:42.933 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/127) 2024-05-04T13:25:43.449 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:25:43.453 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:25:43.453 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:26:09.136 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:26:09.140 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:26:39.243 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:26:39.246 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:27:09.348 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:27:09.352 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:27:39.480 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:27:39.483 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:28:09.550 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:28:09.553 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:28:39.600 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:28:39.603 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:29:09.638 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:29:09.641 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:29:39.752 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:29:39.755 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:30:09.875 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:30:09.878 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:30:39.997 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:30:40.000 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:31:10.056 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:31:10.059 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:31:40.118 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:31:40.121 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:32:10.224 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:32:10.227 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:32:40.306 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:32:40.309 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:33:10.490 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:33:10.493 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:33:40.551 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:33:40.554 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:34:10.865 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:34:10.869 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:34:40.912 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:34:40.916 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:35:11.026 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:35:11.029 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:35:41.149 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:35:41.152 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:36:11.491 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:36:11.494 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:36:41.536 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:36:41.540 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:37:11.659 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:37:11.662 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:37:11.681 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T13:37:11.682+0000 7f7b9124a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 156134) UID: 0 2024-05-04T13:37:11.682 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T13:37:11.682+0000 7fd3bf2df640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 156134) UID: 0 2024-05-04T13:37:11.682 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T13:37:11.682+0000 7f17989d7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 156134) UID: 0 2024-05-04T13:37:11.682 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T13:37:11.682+0000 7f3c99d9c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 156134) UID: 0 2024-05-04T13:39:51.693 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:39:51.696 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:40:21.741 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:40:21.744 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:40:51.841 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:40:51.844 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:41:21.887 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:41:21.891 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:41:51.943 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:41:51.946 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:42:21.991 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:42:21.994 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:42:52.055 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:42:52.058 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:43:22.153 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:43:22.156 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:43:52.291 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:43:52.294 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:44:22.414 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:44:22.417 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:44:52.571 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:44:52.574 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:45:22.876 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:45:22.879 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:45:52.923 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:45:52.926 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:46:23.029 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:46:23.033 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:46:53.274 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:46:53.277 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:47:23.325 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:47:23.329 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:47:53.581 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:47:53.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:48:23.836 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:48:23.839 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:48:53.893 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:48:53.896 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:49:23.939 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:49:23.942 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:49:54.120 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:49:54.124 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:50:24.169 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:50:24.172 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:50:54.214 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:50:54.218 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:51:24.312 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:51:24.315 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:51:54.356 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:51:54.359 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:52:24.399 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:52:24.402 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:52:54.455 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:52:54.458 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:53:24.518 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:53:24.521 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:53:54.633 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:53:54.637 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:54:24.681 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:54:24.685 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:54:54.756 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:54:54.759 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:55:24.799 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:55:24.803 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:55:44.314 INFO:teuthology.orchestra.run.smithi052.stdout:generic/127 1801s 2024-05-04T13:55:44.321 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/127 2024-05-04T13:55:44.322 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:55:44.325 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:55:44.337 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:55:44.338 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/128) 2024-05-04T13:55:44.878 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:55:44.882 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:55:44.882 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:55:44.987 INFO:teuthology.orchestra.run.smithi052.stdout:generic/128 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:55:44.991 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/128 2024-05-04T13:55:44.991 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/128 2024-05-04T13:55:44.991 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:55:44.993 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:55:45.001 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:55:45.001 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/129) 2024-05-04T13:55:45.157 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:55:45.162 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:55:45.162 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:55:45.266 INFO:teuthology.orchestra.run.smithi052.stdout:generic/129 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:55:45.270 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/129 2024-05-04T13:55:45.270 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/129 2024-05-04T13:55:45.270 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:55:45.272 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:55:45.280 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:55:45.281 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/130) 2024-05-04T13:55:45.435 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:55:45.439 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:55:45.440 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:55:45.541 INFO:teuthology.orchestra.run.smithi052.stdout:generic/130 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:55:45.545 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/130 2024-05-04T13:55:45.545 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/130 2024-05-04T13:55:45.545 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:55:45.547 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:55:45.555 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:55:45.556 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/131) 2024-05-04T13:55:45.708 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:55:45.712 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:55:45.712 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:55:47.103 INFO:teuthology.orchestra.run.smithi052.stdout:generic/131 2s 2024-05-04T13:55:47.110 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/131 2024-05-04T13:55:47.110 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:55:47.114 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:55:47.124 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:55:47.125 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/132) 2024-05-04T13:55:47.668 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:55:47.673 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:55:47.673 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:55:47.775 INFO:teuthology.orchestra.run.smithi052.stdout:generic/132 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T13:55:47.778 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/132 2024-05-04T13:55:47.778 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/132 2024-05-04T13:55:47.778 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T13:55:47.780 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:55:47.788 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T13:55:47.788 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/133) 2024-05-04T13:55:47.940 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T13:55:47.944 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T13:55:47.944 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T13:55:54.915 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:55:54.918 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:56:24.995 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:56:24.998 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:56:55.121 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:56:55.126 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:57:25.232 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:57:25.235 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:57:55.297 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:57:55.300 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:58:25.556 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:58:25.559 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:58:55.616 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:58:55.622 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:59:25.747 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:59:25.750 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:59:55.819 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T13:59:55.824 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:00:25.875 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:00:25.879 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:00:55.948 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:00:55.951 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:01:26.003 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:01:26.006 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:01:56.126 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:01:56.130 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:02:26.249 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:02:26.255 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:02:56.321 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:02:56.324 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:03:26.368 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:03:26.371 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:03:56.708 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:03:56.711 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:04:26.755 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:04:26.759 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:04:32.364 INFO:teuthology.orchestra.run.smithi052.stdout:generic/133 525s 2024-05-04T14:04:32.372 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/133 2024-05-04T14:04:32.372 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:32.375 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:32.540 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:32.540 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/134) 2024-05-04T14:04:32.958 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:32.962 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:32.962 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:33.133 INFO:teuthology.orchestra.run.smithi052.stdout:generic/134 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:33.137 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/134 2024-05-04T14:04:33.137 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/134 2024-05-04T14:04:33.137 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:33.140 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:33.156 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:33.156 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/135) 2024-05-04T14:04:33.774 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:33.778 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:33.778 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:33.945 INFO:teuthology.orchestra.run.smithi052.stdout:generic/135 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:33.949 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/135 2024-05-04T14:04:33.949 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/135 2024-05-04T14:04:33.949 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:33.952 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:33.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:33.960 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/136) 2024-05-04T14:04:34.115 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:34.119 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:34.119 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:34.278 INFO:teuthology.orchestra.run.smithi052.stdout:generic/136 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:34.282 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/136 2024-05-04T14:04:34.282 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/136 2024-05-04T14:04:34.282 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:34.284 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:34.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:34.292 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/137) 2024-05-04T14:04:34.790 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:34.794 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:34.794 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:34.952 INFO:teuthology.orchestra.run.smithi052.stdout:generic/137 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:34.955 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/137 2024-05-04T14:04:34.955 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/137 2024-05-04T14:04:34.955 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:34.957 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:34.965 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:34.965 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/138) 2024-05-04T14:04:35.465 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:35.468 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:35.469 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:35.619 INFO:teuthology.orchestra.run.smithi052.stdout:generic/138 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:35.622 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/138 2024-05-04T14:04:35.622 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/138 2024-05-04T14:04:35.622 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:35.624 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:35.633 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:35.634 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/139) 2024-05-04T14:04:36.111 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:36.115 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:36.116 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:36.268 INFO:teuthology.orchestra.run.smithi052.stdout:generic/139 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:36.272 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/139 2024-05-04T14:04:36.272 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/139 2024-05-04T14:04:36.272 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:36.274 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:36.282 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:36.282 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/140) 2024-05-04T14:04:36.784 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:36.788 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:36.789 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:36.941 INFO:teuthology.orchestra.run.smithi052.stdout:generic/140 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:36.944 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/140 2024-05-04T14:04:36.944 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/140 2024-05-04T14:04:36.944 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:36.946 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:36.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:36.954 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/141) 2024-05-04T14:04:37.479 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:37.483 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:37.483 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:37.585 INFO:teuthology.orchestra.run.smithi052.stdout:generic/141 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:37.589 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/141 2024-05-04T14:04:37.589 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/141 2024-05-04T14:04:37.589 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:37.591 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:37.599 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:37.599 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/142) 2024-05-04T14:04:37.751 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:37.755 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:37.756 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:37.907 INFO:teuthology.orchestra.run.smithi052.stdout:generic/142 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:37.910 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/142 2024-05-04T14:04:37.910 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/142 2024-05-04T14:04:37.910 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:37.913 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:37.920 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:37.920 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/143) 2024-05-04T14:04:38.424 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:38.428 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:38.429 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:38.580 INFO:teuthology.orchestra.run.smithi052.stdout:generic/143 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:38.584 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/143 2024-05-04T14:04:38.584 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/143 2024-05-04T14:04:38.584 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:38.587 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:38.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:38.595 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/144) 2024-05-04T14:04:39.077 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:39.081 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:39.082 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:39.231 INFO:teuthology.orchestra.run.smithi052.stdout:generic/144 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:39.235 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/144 2024-05-04T14:04:39.235 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/144 2024-05-04T14:04:39.235 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:39.237 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:39.245 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:39.245 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/145) 2024-05-04T14:04:39.742 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:39.746 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:39.746 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:39.900 INFO:teuthology.orchestra.run.smithi052.stdout:generic/145 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:39.903 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/145 2024-05-04T14:04:39.903 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/145 2024-05-04T14:04:39.903 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:39.906 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:39.913 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:39.913 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/146) 2024-05-04T14:04:40.424 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:40.428 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:40.428 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:40.582 INFO:teuthology.orchestra.run.smithi052.stdout:generic/146 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:40.585 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/146 2024-05-04T14:04:40.586 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/146 2024-05-04T14:04:40.586 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:40.588 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:40.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:40.595 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/147) 2024-05-04T14:04:41.090 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:41.094 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:41.094 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:41.246 INFO:teuthology.orchestra.run.smithi052.stdout:generic/147 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:41.249 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/147 2024-05-04T14:04:41.249 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/147 2024-05-04T14:04:41.249 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:41.251 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:41.259 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:41.259 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/148) 2024-05-04T14:04:41.766 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:41.771 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:41.771 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:41.923 INFO:teuthology.orchestra.run.smithi052.stdout:generic/148 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:41.926 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/148 2024-05-04T14:04:41.927 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/148 2024-05-04T14:04:41.927 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:41.929 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:41.937 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:41.937 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/149) 2024-05-04T14:04:42.455 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:42.459 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:42.459 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:42.607 INFO:teuthology.orchestra.run.smithi052.stdout:generic/149 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:42.611 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/149 2024-05-04T14:04:42.611 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/149 2024-05-04T14:04:42.611 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:42.613 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:42.621 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:42.621 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/150) 2024-05-04T14:04:43.107 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:43.111 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:43.111 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:43.266 INFO:teuthology.orchestra.run.smithi052.stdout:generic/150 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:43.269 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/150 2024-05-04T14:04:43.269 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/150 2024-05-04T14:04:43.270 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:43.272 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:43.279 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:43.279 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/151) 2024-05-04T14:04:43.786 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:43.790 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:43.790 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:43.938 INFO:teuthology.orchestra.run.smithi052.stdout:generic/151 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:43.941 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/151 2024-05-04T14:04:43.942 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/151 2024-05-04T14:04:43.942 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:43.943 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:43.951 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:43.952 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/152) 2024-05-04T14:04:44.462 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:44.466 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:44.466 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:44.619 INFO:teuthology.orchestra.run.smithi052.stdout:generic/152 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:44.622 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/152 2024-05-04T14:04:44.622 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/152 2024-05-04T14:04:44.622 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:44.624 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:44.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:44.634 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/153) 2024-05-04T14:04:45.120 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:45.124 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:45.124 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:45.274 INFO:teuthology.orchestra.run.smithi052.stdout:generic/153 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:45.278 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/153 2024-05-04T14:04:45.278 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/153 2024-05-04T14:04:45.278 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:45.280 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:45.288 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:45.288 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/154) 2024-05-04T14:04:45.772 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:45.776 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:45.776 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:45.930 INFO:teuthology.orchestra.run.smithi052.stdout:generic/154 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:45.933 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/154 2024-05-04T14:04:45.933 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/154 2024-05-04T14:04:45.933 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:45.936 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:45.943 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:45.944 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/155) 2024-05-04T14:04:46.420 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:46.424 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:46.424 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:46.575 INFO:teuthology.orchestra.run.smithi052.stdout:generic/155 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:46.578 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/155 2024-05-04T14:04:46.579 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/155 2024-05-04T14:04:46.579 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:46.581 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:46.588 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:46.589 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/156) 2024-05-04T14:04:47.074 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:47.078 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:47.352 INFO:teuthology.orchestra.run.smithi052.stdout:generic/156 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:47.356 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/156 2024-05-04T14:04:47.356 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/156 2024-05-04T14:04:47.356 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:47.359 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:47.367 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:47.367 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/157) 2024-05-04T14:04:47.866 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:47.870 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:47.870 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:48.033 INFO:teuthology.orchestra.run.smithi052.stdout:generic/157 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:48.036 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/157 2024-05-04T14:04:48.036 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/157 2024-05-04T14:04:48.036 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:48.038 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:48.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:48.046 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/158) 2024-05-04T14:04:48.547 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:48.551 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:48.551 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:48.722 INFO:teuthology.orchestra.run.smithi052.stdout:generic/158 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:48.726 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/158 2024-05-04T14:04:48.726 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/158 2024-05-04T14:04:48.726 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:48.728 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:48.735 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:48.736 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/159) 2024-05-04T14:04:49.220 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:49.224 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:49.224 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:49.388 INFO:teuthology.orchestra.run.smithi052.stdout:generic/159 [not run] file system doesn't support chattr +i 2024-05-04T14:04:49.392 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/159 2024-05-04T14:04:49.392 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/159 2024-05-04T14:04:49.392 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:49.394 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:49.402 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:49.402 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/160) 2024-05-04T14:04:49.552 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:49.556 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:49.556 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:49.717 INFO:teuthology.orchestra.run.smithi052.stdout:generic/160 [not run] file system doesn't support chattr +i 2024-05-04T14:04:49.721 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/160 2024-05-04T14:04:49.721 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/160 2024-05-04T14:04:49.722 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:49.723 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:49.731 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:49.732 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/161) 2024-05-04T14:04:49.886 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:49.891 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:49.891 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:49.993 INFO:teuthology.orchestra.run.smithi052.stdout:generic/161 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:49.999 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/161 2024-05-04T14:04:49.999 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/161 2024-05-04T14:04:49.999 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:49.999 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:50.008 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:50.008 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/162) 2024-05-04T14:04:50.163 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:50.167 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:50.167 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:50.267 INFO:teuthology.orchestra.run.smithi052.stdout:generic/162 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:50.271 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/162 2024-05-04T14:04:50.271 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/162 2024-05-04T14:04:50.271 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:50.273 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:50.281 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:50.281 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/163) 2024-05-04T14:04:50.432 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:50.436 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:50.436 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:50.539 INFO:teuthology.orchestra.run.smithi052.stdout:generic/163 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:50.542 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/163 2024-05-04T14:04:50.543 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/163 2024-05-04T14:04:50.543 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:50.545 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:50.553 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:50.553 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/164) 2024-05-04T14:04:50.710 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:50.715 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:50.715 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:50.830 INFO:teuthology.orchestra.run.smithi052.stdout:generic/164 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:50.833 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/164 2024-05-04T14:04:50.833 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/164 2024-05-04T14:04:50.833 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:50.835 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:50.843 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:50.843 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/165) 2024-05-04T14:04:50.998 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:51.002 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:51.002 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:51.106 INFO:teuthology.orchestra.run.smithi052.stdout:generic/165 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:51.110 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/165 2024-05-04T14:04:51.110 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/165 2024-05-04T14:04:51.110 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:51.112 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:51.120 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:51.120 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/166) 2024-05-04T14:04:51.272 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:51.276 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:51.276 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:51.381 INFO:teuthology.orchestra.run.smithi052.stdout:generic/166 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:51.385 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/166 2024-05-04T14:04:51.385 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/166 2024-05-04T14:04:51.385 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:51.387 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:51.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:51.395 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/167) 2024-05-04T14:04:51.550 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:51.554 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:51.554 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:51.658 INFO:teuthology.orchestra.run.smithi052.stdout:generic/167 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:51.661 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/167 2024-05-04T14:04:51.661 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/167 2024-05-04T14:04:51.661 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:51.664 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:51.671 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:51.672 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/168) 2024-05-04T14:04:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:51.832 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:51.832 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:generic/168 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:51.948 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/168 2024-05-04T14:04:51.948 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/168 2024-05-04T14:04:51.948 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:51.951 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:51.958 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:51.958 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/169) 2024-05-04T14:04:52.112 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:52.116 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:52.116 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:52.218 INFO:teuthology.orchestra.run.smithi052.stdout:generic/169 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:52.222 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/169 2024-05-04T14:04:52.222 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/169 2024-05-04T14:04:52.222 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:52.224 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:52.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:52.233 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/170) 2024-05-04T14:04:52.388 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:52.392 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:52.392 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:52.493 INFO:teuthology.orchestra.run.smithi052.stdout:generic/170 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:52.497 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/170 2024-05-04T14:04:52.497 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/170 2024-05-04T14:04:52.497 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:52.499 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:52.507 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:52.507 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/171) 2024-05-04T14:04:52.661 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:52.665 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:52.665 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:52.793 INFO:teuthology.orchestra.run.smithi052.stdout:generic/171 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:52.797 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/171 2024-05-04T14:04:52.797 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/171 2024-05-04T14:04:52.797 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:52.799 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:52.810 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:52.810 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/172) 2024-05-04T14:04:52.961 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:52.965 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:52.965 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:53.071 INFO:teuthology.orchestra.run.smithi052.stdout:generic/172 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:53.074 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/172 2024-05-04T14:04:53.074 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/172 2024-05-04T14:04:53.074 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:53.076 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:53.084 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:53.084 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/173) 2024-05-04T14:04:53.236 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:53.240 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:53.240 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:53.347 INFO:teuthology.orchestra.run.smithi052.stdout:generic/173 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:53.351 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/173 2024-05-04T14:04:53.351 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/173 2024-05-04T14:04:53.351 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:53.353 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:53.361 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:53.361 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/174) 2024-05-04T14:04:53.513 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:53.517 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:53.518 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:53.637 INFO:teuthology.orchestra.run.smithi052.stdout:generic/174 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:53.640 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/174 2024-05-04T14:04:53.641 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/174 2024-05-04T14:04:53.641 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:53.644 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:53.652 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:53.652 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/175) 2024-05-04T14:04:53.808 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:53.812 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:53.812 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:53.917 INFO:teuthology.orchestra.run.smithi052.stdout:generic/175 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:53.921 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/175 2024-05-04T14:04:53.921 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/175 2024-05-04T14:04:53.921 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:53.923 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:53.931 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:53.931 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/176) 2024-05-04T14:04:54.085 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:54.089 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:54.089 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:54.194 INFO:teuthology.orchestra.run.smithi052.stdout:generic/176 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:54.197 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/176 2024-05-04T14:04:54.198 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/176 2024-05-04T14:04:54.198 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:54.200 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:54.208 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:54.208 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/177) 2024-05-04T14:04:54.374 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:54.378 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:54.378 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:54.500 INFO:teuthology.orchestra.run.smithi052.stdout:generic/177 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:54.503 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/177 2024-05-04T14:04:54.504 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/177 2024-05-04T14:04:54.504 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:54.506 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:54.513 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:54.513 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/178) 2024-05-04T14:04:54.667 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:54.671 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:54.671 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:54.830 INFO:teuthology.orchestra.run.smithi052.stdout:generic/178 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:54.834 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/178 2024-05-04T14:04:54.834 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/178 2024-05-04T14:04:54.834 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:54.836 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:54.844 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:54.844 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/179) 2024-05-04T14:04:55.332 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:55.337 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:55.337 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:55.495 INFO:teuthology.orchestra.run.smithi052.stdout:generic/179 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:55.498 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/179 2024-05-04T14:04:55.498 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/179 2024-05-04T14:04:55.498 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:55.501 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:55.508 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:55.509 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/180) 2024-05-04T14:04:55.994 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:55.999 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:55.999 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:56.155 INFO:teuthology.orchestra.run.smithi052.stdout:generic/180 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:56.159 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/180 2024-05-04T14:04:56.159 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/180 2024-05-04T14:04:56.159 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:56.161 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:56.171 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:56.171 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/181) 2024-05-04T14:04:56.672 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:56.676 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:56.829 INFO:teuthology.orchestra.run.smithi052.stdout:generic/181 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:56.833 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/181 2024-05-04T14:04:56.833 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/181 2024-05-04T14:04:56.833 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:56.835 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:56.843 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:56.843 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/182) 2024-05-04T14:04:56.919 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:04:56.921 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:04:57.411 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:57.459 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:57.459 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:57.573 INFO:teuthology.orchestra.run.smithi052.stdout:generic/182 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-05-04T14:04:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/182 2024-05-04T14:04:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/182 2024-05-04T14:04:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:57.579 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:57.587 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:57.587 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/183) 2024-05-04T14:04:58.085 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:58.089 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:58.089 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:58.190 INFO:teuthology.orchestra.run.smithi052.stdout:generic/183 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:58.193 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/183 2024-05-04T14:04:58.193 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/183 2024-05-04T14:04:58.193 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:58.195 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:58.204 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:58.204 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/184) 2024-05-04T14:04:58.355 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:58.359 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:58.359 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:58.502 INFO:teuthology.orchestra.run.smithi052.stdout:generic/184 [failed, exit status 1]- output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/184.out.bad) 2024-05-04T14:04:58.504 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/184.out 2024-05-04 12:19:31.968992880 +0000 2024-05-04T14:04:58.504 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/184.out.bad 2024-05-04 14:04:58.502549420 +0000 2024-05-04T14:04:58.504 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -1 +1,2 @@ 2024-05-04T14:04:58.504 INFO:teuthology.orchestra.run.smithi052.stdout: QA output created by 184 2024-05-04T14:04:58.504 INFO:teuthology.orchestra.run.smithi052.stdout: +/tmp/tmp.BtfdD8Y1eWxfstests-dev/tests/generic/184: line 23: /tmp/tmp.FGpMFsgr0Qtest/null: Permission denied 2024-05-04T14:04:58.507 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/184 2024-05-04T14:04:58.507 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/184 2024-05-04T14:04:58.507 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T14:04:58.509 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:58.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T14:04:58.517 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T14:04:58.517 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T14:04:58.517 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/185) 2024-05-04T14:04:58.667 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:58.671 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:58.671 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:58.773 INFO:teuthology.orchestra.run.smithi052.stdout:generic/185 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:58.777 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/185 2024-05-04T14:04:58.777 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/185 2024-05-04T14:04:58.777 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:58.779 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:58.787 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:58.787 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/186) 2024-05-04T14:04:58.942 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:58.946 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:58.946 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:59.049 INFO:teuthology.orchestra.run.smithi052.stdout:generic/186 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:59.052 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/186 2024-05-04T14:04:59.052 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/186 2024-05-04T14:04:59.052 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:59.054 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:59.062 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:59.062 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/187) 2024-05-04T14:04:59.216 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:59.220 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:59.220 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:59.329 INFO:teuthology.orchestra.run.smithi052.stdout:generic/187 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:59.333 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/187 2024-05-04T14:04:59.333 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/187 2024-05-04T14:04:59.333 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:59.336 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:59.343 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:59.344 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/188) 2024-05-04T14:04:59.496 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:59.500 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:59.501 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:59.602 INFO:teuthology.orchestra.run.smithi052.stdout:generic/188 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:59.606 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/188 2024-05-04T14:04:59.606 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/188 2024-05-04T14:04:59.606 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:59.608 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:59.616 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:59.616 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/189) 2024-05-04T14:04:59.766 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:04:59.770 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:04:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:59.872 INFO:teuthology.orchestra.run.smithi052.stdout:generic/189 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:04:59.876 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/189 2024-05-04T14:04:59.876 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/189 2024-05-04T14:04:59.876 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:04:59.878 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:04:59.886 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:04:59.886 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/190) 2024-05-04T14:05:00.038 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:00.042 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:00.042 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:00.143 INFO:teuthology.orchestra.run.smithi052.stdout:generic/190 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:05:00.146 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/190 2024-05-04T14:05:00.146 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/190 2024-05-04T14:05:00.146 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:00.158 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:00.158 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/191) 2024-05-04T14:05:00.309 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:00.313 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:00.416 INFO:teuthology.orchestra.run.smithi052.stdout:generic/191 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:05:00.419 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/191 2024-05-04T14:05:00.419 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/191 2024-05-04T14:05:00.419 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:00.429 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:00.429 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/192) 2024-05-04T14:05:00.584 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:00.588 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:00.697 INFO:teuthology.orchestra.run.smithi052.stdout:generic/192 [not run] atime not maintained by ceph-fuse 2024-05-04T14:05:00.700 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/192 2024-05-04T14:05:00.700 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/192 2024-05-04T14:05:00.700 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:00.711 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:00.711 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/193) 2024-05-04T14:05:00.864 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:00.868 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:01.547 INFO:teuthology.orchestra.run.smithi052.stdout:generic/193 - output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/193.out.bad) 2024-05-04T14:05:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/193.out 2024-05-04 12:19:31.972992822 +0000 2024-05-04T14:05:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/193.out.bad 2024-05-04 14:05:01.546510841 +0000 2024-05-04T14:05:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -24,7 +24,6 @@ 2024-05-04T14:05:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: user: chmod a+r on qa_user owned file (should succeed) 2024-05-04T14:05:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: user: chmod a+r on root owned file (should fail) 2024-05-04T14:05:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: -chmod: changing permissions of 'test.root': Operation not permitted 2024-05-04T14:05:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: check that the sgid bit is cleared 2024-05-04T14:05:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: -rw-rw-rw- 2024-05-04T14:05:01.550 INFO:teuthology.orchestra.run.smithi052.stdout: check that suid bit is not cleared 2024-05-04T14:05:01.552 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/193 2024-05-04T14:05:01.552 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/193 2024-05-04T14:05:01.552 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T14:05:01.554 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:01.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T14:05:01.562 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T14:05:01.562 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T14:05:01.562 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/194) 2024-05-04T14:05:02.039 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:02.043 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:02.043 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:02.157 INFO:teuthology.orchestra.run.smithi052.stdout:generic/194 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:05:02.161 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/194 2024-05-04T14:05:02.161 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/194 2024-05-04T14:05:02.161 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:02.164 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:02.176 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:02.176 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/195) 2024-05-04T14:05:02.325 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:02.329 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:02.329 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:02.446 INFO:teuthology.orchestra.run.smithi052.stdout:generic/195 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:05:02.447 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/195 2024-05-04T14:05:02.447 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/195 2024-05-04T14:05:02.447 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:02.449 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:02.458 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:02.458 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/196) 2024-05-04T14:05:02.608 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:02.612 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:02.612 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:02.720 INFO:teuthology.orchestra.run.smithi052.stdout:generic/196 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:05:02.725 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/196 2024-05-04T14:05:02.725 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/196 2024-05-04T14:05:02.725 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:02.728 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:02.736 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:02.737 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/197) 2024-05-04T14:05:02.884 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:02.888 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:02.888 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:02.990 INFO:teuthology.orchestra.run.smithi052.stdout:generic/197 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:05:02.994 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/197 2024-05-04T14:05:02.994 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/197 2024-05-04T14:05:02.994 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:02.996 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:03.005 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:03.005 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/198) 2024-05-04T14:05:03.151 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:03.155 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:03.155 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:04.128 INFO:teuthology.orchestra.run.smithi052.stdout:generic/198 1s 2024-05-04T14:05:04.136 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/198 2024-05-04T14:05:04.136 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:04.139 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:04.152 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:04.152 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/199) 2024-05-04T14:05:04.655 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:04.659 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:04.659 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:04.760 INFO:teuthology.orchestra.run.smithi052.stdout:generic/199 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:05:04.764 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/199 2024-05-04T14:05:04.764 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/199 2024-05-04T14:05:04.764 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:04.766 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:04.775 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:04.775 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/200) 2024-05-04T14:05:04.920 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:04.924 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:04.924 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:05.025 INFO:teuthology.orchestra.run.smithi052.stdout:generic/200 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:05:05.029 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/200 2024-05-04T14:05:05.029 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/200 2024-05-04T14:05:05.029 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:05.031 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:05.039 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:05.039 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/201) 2024-05-04T14:05:05.209 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:05.210 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:05.210 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:05.326 INFO:teuthology.orchestra.run.smithi052.stdout:generic/201 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:05:05.330 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/201 2024-05-04T14:05:05.330 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/201 2024-05-04T14:05:05.330 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:05.332 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:05.340 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:05.340 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/202) 2024-05-04T14:05:05.485 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:05.489 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:05.489 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:05.591 INFO:teuthology.orchestra.run.smithi052.stdout:generic/202 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:05:05.595 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/202 2024-05-04T14:05:05.595 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/202 2024-05-04T14:05:05.595 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:05.597 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:05.605 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:05.605 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/203) 2024-05-04T14:05:05.750 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:05.755 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:05.755 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:05.858 INFO:teuthology.orchestra.run.smithi052.stdout:generic/203 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:05:05.862 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/203 2024-05-04T14:05:05.862 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/203 2024-05-04T14:05:05.862 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:05.864 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:05.872 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:05.872 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/204) 2024-05-04T14:05:06.020 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:06.025 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:06.025 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:07.605 INFO:teuthology.orchestra.run.smithi052.stdout:generic/204 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:05:07.612 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/204 2024-05-04T14:05:07.612 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/204 2024-05-04T14:05:07.612 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:07.616 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:07.663 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:07.663 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/205) 2024-05-04T14:05:07.866 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:07.871 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:07.871 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:08.012 INFO:teuthology.orchestra.run.smithi052.stdout:generic/205 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:05:08.016 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/205 2024-05-04T14:05:08.016 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/205 2024-05-04T14:05:08.017 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:08.019 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:08.029 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:08.030 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/206) 2024-05-04T14:05:08.206 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:08.211 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:08.211 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:08.349 INFO:teuthology.orchestra.run.smithi052.stdout:generic/206 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:05:08.353 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/206 2024-05-04T14:05:08.353 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/206 2024-05-04T14:05:08.353 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:08.356 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:08.366 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:08.366 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/207) 2024-05-04T14:05:08.537 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:08.541 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:08.541 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:27.376 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:05:27.379 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:05:49.405 INFO:teuthology.orchestra.run.smithi052.stdout:generic/207 41s 2024-05-04T14:05:49.413 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/207 2024-05-04T14:05:49.413 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:05:49.416 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:49.624 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:05:49.625 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/208) 2024-05-04T14:05:49.966 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:05:49.970 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:05:49.970 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:05:57.423 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:05:57.426 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:06:27.528 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:06:27.532 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:06:57.616 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:06:57.620 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:07:27.940 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:07:27.943 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:07:58.121 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:07:58.124 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:08:28.227 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:08:28.230 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:08:58.351 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:08:58.354 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:09:10.365 INFO:teuthology.orchestra.run.smithi052.stdout:generic/208 201s 2024-05-04T14:09:10.372 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/208 2024-05-04T14:09:10.372 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:10.375 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:10.397 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:10.398 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/209) 2024-05-04T14:09:10.903 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:10.907 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:10.907 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:28.446 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:09:28.450 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:09:30.093 INFO:teuthology.orchestra.run.smithi052.stdout:generic/209 [failed, exit status 1]- output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/209.out.bad) 2024-05-04T14:09:30.137 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/209.out 2024-05-04 12:19:31.972992822 +0000 2024-05-04T14:09:30.137 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/209.out.bad 2024-05-04 14:09:30.091038223 +0000 2024-05-04T14:09:30.137 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -1,2 +1,2 @@ 2024-05-04T14:09:30.137 INFO:teuthology.orchestra.run.smithi052.stdout: QA output created by 209 2024-05-04T14:09:30.137 INFO:teuthology.orchestra.run.smithi052.stdout: -test ran for 30 seconds without error 2024-05-04T14:09:30.137 INFO:teuthology.orchestra.run.smithi052.stdout: +reader found old byte at pos 4096 2024-05-04T14:09:30.137 INFO:teuthology.orchestra.run.smithi052.stdout: \ No newline at end of file 2024-05-04T14:09:30.161 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/209 2024-05-04T14:09:30.161 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/209 2024-05-04T14:09:30.161 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T14:09:30.172 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:30.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T14:09:30.214 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T14:09:30.214 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T14:09:30.215 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/210) 2024-05-04T14:09:30.425 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:30.429 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:30.429 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:30.801 INFO:teuthology.orchestra.run.smithi052.stdout:generic/210 0s 2024-05-04T14:09:30.808 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/210 2024-05-04T14:09:30.808 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:30.812 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:30.849 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:30.850 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/211) 2024-05-04T14:09:31.315 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:31.320 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:31.320 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:31.696 INFO:teuthology.orchestra.run.smithi052.stdout:generic/211 0s 2024-05-04T14:09:31.702 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/211 2024-05-04T14:09:31.702 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:31.705 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:31.715 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:31.716 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/212) 2024-05-04T14:09:31.871 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:31.875 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:31.875 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:32.109 INFO:teuthology.orchestra.run.smithi052.stdout:generic/212 1s 2024-05-04T14:09:32.116 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/212 2024-05-04T14:09:32.116 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:32.119 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:32.128 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:32.128 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/213) 2024-05-04T14:09:32.281 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:32.285 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:32.285 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:32.695 INFO:teuthology.orchestra.run.smithi052.stdout:generic/213 - output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/213.out.bad) 2024-05-04T14:09:32.697 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/213.out 2024-05-04 12:19:31.972992822 +0000 2024-05-04T14:09:32.697 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/213.out.bad 2024-05-04 14:09:32.694999576 +0000 2024-05-04T14:09:32.697 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -1,4 +1,3 @@ 2024-05-04T14:09:32.697 INFO:teuthology.orchestra.run.smithi052.stdout: QA output created by 213 2024-05-04T14:09:32.697 INFO:teuthology.orchestra.run.smithi052.stdout: We should get: fallocate: No space left on device 2024-05-04T14:09:32.697 INFO:teuthology.orchestra.run.smithi052.stdout: Strangely, xfs_io sometimes says "Success" when something went wrong, FYI 2024-05-04T14:09:32.697 INFO:teuthology.orchestra.run.smithi052.stdout: -fallocate: No space left on device 2024-05-04T14:09:32.699 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/213 2024-05-04T14:09:32.699 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/213 2024-05-04T14:09:32.699 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T14:09:32.701 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:32.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T14:09:32.711 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T14:09:32.711 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T14:09:32.711 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/214) 2024-05-04T14:09:33.202 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:33.206 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:33.206 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:33.652 INFO:teuthology.orchestra.run.smithi052.stdout:generic/214 0s 2024-05-04T14:09:33.658 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/214 2024-05-04T14:09:33.658 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:33.660 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:33.676 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:33.677 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/215) 2024-05-04T14:09:34.242 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:34.251 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:34.251 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:37.330 INFO:teuthology.orchestra.run.smithi052.stdout:generic/215 3s 2024-05-04T14:09:37.336 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/215 2024-05-04T14:09:37.336 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:37.338 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:37.360 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:37.360 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/216) 2024-05-04T14:09:37.583 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:37.590 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:37.590 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:37.749 INFO:teuthology.orchestra.run.smithi052.stdout:generic/216 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:37.754 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/216 2024-05-04T14:09:37.754 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/216 2024-05-04T14:09:37.754 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:37.757 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:37.769 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:37.769 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/217) 2024-05-04T14:09:38.041 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:38.046 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:38.046 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:38.206 INFO:teuthology.orchestra.run.smithi052.stdout:generic/217 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:38.210 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/217 2024-05-04T14:09:38.210 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/217 2024-05-04T14:09:38.211 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:38.212 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:38.224 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:38.224 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/218) 2024-05-04T14:09:38.491 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:38.495 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:38.495 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:38.660 INFO:teuthology.orchestra.run.smithi052.stdout:generic/218 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:38.664 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/218 2024-05-04T14:09:38.664 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/218 2024-05-04T14:09:38.665 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:38.683 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:38.683 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/219) 2024-05-04T14:09:38.894 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:38.900 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:38.900 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:39.055 INFO:teuthology.orchestra.run.smithi052.stdout:generic/219 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:39.061 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/219 2024-05-04T14:09:39.061 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/219 2024-05-04T14:09:39.061 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:39.063 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:39.073 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:39.073 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/220) 2024-05-04T14:09:39.330 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:39.338 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:39.338 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:39.483 INFO:teuthology.orchestra.run.smithi052.stdout:generic/220 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:39.487 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/220 2024-05-04T14:09:39.487 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/220 2024-05-04T14:09:39.487 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:39.489 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:39.504 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:39.504 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/221) 2024-05-04T14:09:39.759 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:39.764 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:39.764 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:40.941 INFO:teuthology.orchestra.run.smithi052.stdout:generic/221 1s 2024-05-04T14:09:40.948 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/221 2024-05-04T14:09:40.948 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:40.950 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:40.961 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:40.961 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/222) 2024-05-04T14:09:41.204 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:41.209 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:41.209 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:41.363 INFO:teuthology.orchestra.run.smithi052.stdout:generic/222 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:41.368 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/222 2024-05-04T14:09:41.368 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/222 2024-05-04T14:09:41.368 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:41.371 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:41.388 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:41.388 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/223) 2024-05-04T14:09:41.614 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:41.619 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:41.619 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:41.766 INFO:teuthology.orchestra.run.smithi052.stdout:generic/223 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:41.770 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/223 2024-05-04T14:09:41.770 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/223 2024-05-04T14:09:41.770 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:41.773 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:41.794 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:41.794 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/224) 2024-05-04T14:09:42.013 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:42.028 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:42.028 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:42.194 INFO:teuthology.orchestra.run.smithi052.stdout:generic/224 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:42.198 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/224 2024-05-04T14:09:42.199 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/224 2024-05-04T14:09:42.199 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:42.209 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:42.231 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:42.231 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/225) 2024-05-04T14:09:42.453 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:42.457 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:42.458 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:42.607 INFO:teuthology.orchestra.run.smithi052.stdout:generic/225 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:42.611 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/225 2024-05-04T14:09:42.611 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/225 2024-05-04T14:09:42.611 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:42.614 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:42.625 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:42.625 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/226) 2024-05-04T14:09:42.913 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:42.923 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:42.923 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:43.074 INFO:teuthology.orchestra.run.smithi052.stdout:generic/226 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:43.079 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/226 2024-05-04T14:09:43.079 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/226 2024-05-04T14:09:43.079 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:43.081 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:43.093 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:43.094 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/227) 2024-05-04T14:09:43.337 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:43.342 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:43.342 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:43.498 INFO:teuthology.orchestra.run.smithi052.stdout:generic/227 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:43.502 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/227 2024-05-04T14:09:43.502 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/227 2024-05-04T14:09:43.502 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:43.505 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:43.522 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:43.522 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/228) 2024-05-04T14:09:43.753 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:43.757 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:43.757 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:44.074 INFO:teuthology.orchestra.run.smithi052.stdout:generic/228 1s 2024-05-04T14:09:44.085 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/228 2024-05-04T14:09:44.085 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:44.087 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:44.160 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:44.161 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/229) 2024-05-04T14:09:44.700 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:44.705 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:44.705 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:44.867 INFO:teuthology.orchestra.run.smithi052.stdout:generic/229 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:44.870 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/229 2024-05-04T14:09:44.871 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/229 2024-05-04T14:09:44.871 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:44.874 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:44.884 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:44.884 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/230) 2024-05-04T14:09:45.117 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:45.121 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:45.121 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:45.280 INFO:teuthology.orchestra.run.smithi052.stdout:generic/230 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:45.285 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/230 2024-05-04T14:09:45.285 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/230 2024-05-04T14:09:45.285 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:45.289 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:45.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:45.310 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/231) 2024-05-04T14:09:45.569 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:45.579 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:45.579 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:45.723 INFO:teuthology.orchestra.run.smithi052.stdout:generic/231 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:45.728 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/231 2024-05-04T14:09:45.728 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/231 2024-05-04T14:09:45.728 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:45.735 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:45.744 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:45.744 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/232) 2024-05-04T14:09:45.987 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:45.992 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:46.140 INFO:teuthology.orchestra.run.smithi052.stdout:generic/232 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:46.145 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/232 2024-05-04T14:09:46.145 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/232 2024-05-04T14:09:46.145 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:46.148 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:46.160 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:46.160 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/233) 2024-05-04T14:09:46.415 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:46.420 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:46.420 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:46.576 INFO:teuthology.orchestra.run.smithi052.stdout:generic/233 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:46.580 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/233 2024-05-04T14:09:46.580 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/233 2024-05-04T14:09:46.581 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:46.584 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:46.600 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:46.601 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/234) 2024-05-04T14:09:46.830 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:46.840 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:46.840 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:46.997 INFO:teuthology.orchestra.run.smithi052.stdout:generic/234 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:47.001 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/234 2024-05-04T14:09:47.002 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/234 2024-05-04T14:09:47.002 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:47.004 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:47.018 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:47.018 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/235) 2024-05-04T14:09:47.306 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:47.311 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:47.311 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:47.485 INFO:teuthology.orchestra.run.smithi052.stdout:generic/235 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:47.489 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/235 2024-05-04T14:09:47.489 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/235 2024-05-04T14:09:47.489 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:47.491 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:47.502 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:47.503 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/236) 2024-05-04T14:09:47.768 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:47.773 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:47.773 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:49.187 INFO:teuthology.orchestra.run.smithi052.stdout:generic/236 2s 2024-05-04T14:09:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/236 2024-05-04T14:09:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:49.201 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:49.269 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:49.270 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/237) 2024-05-04T14:09:49.824 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:49.829 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:49.829 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:50.079 INFO:teuthology.orchestra.run.smithi052.stdout:generic/237 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-05-04T14:09:50.088 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/237 2024-05-04T14:09:50.089 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/237 2024-05-04T14:09:50.089 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:50.092 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:50.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:50.150 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/238) 2024-05-04T14:09:50.395 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:50.399 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:50.399 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:50.550 INFO:teuthology.orchestra.run.smithi052.stdout:generic/238 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:09:50.554 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/238 2024-05-04T14:09:50.555 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/238 2024-05-04T14:09:50.555 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:50.557 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:50.567 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:50.567 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/239) 2024-05-04T14:09:50.829 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:50.833 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:50.833 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:51.007 INFO:teuthology.orchestra.run.smithi052.stdout:generic/239 [not run] Sparse files are not supported or do not work as expected 2024-05-04T14:09:51.011 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/239 2024-05-04T14:09:51.011 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/239 2024-05-04T14:09:51.011 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:51.018 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:51.034 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:51.034 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/240) 2024-05-04T14:09:51.275 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:51.283 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:51.283 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:51.490 INFO:teuthology.orchestra.run.smithi052.stdout:generic/240 [not run] Sparse files are not supported or do not work as expected 2024-05-04T14:09:51.494 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/240 2024-05-04T14:09:51.494 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/240 2024-05-04T14:09:51.494 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:09:51.497 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:51.509 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:09:51.509 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/241) 2024-05-04T14:09:51.768 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:09:51.773 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:09:51.773 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:09:58.560 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:09:58.563 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:10:28.692 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:10:28.694 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:10:58.867 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:10:58.869 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:11:05.434 INFO:teuthology.orchestra.run.smithi052.stdout:generic/241 74s 2024-05-04T14:11:05.440 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/241 2024-05-04T14:11:05.441 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:11:05.443 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:11:05.532 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:11:05.532 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/242) 2024-05-04T14:11:06.080 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:11:06.085 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:11:06.085 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:11:06.501 INFO:teuthology.orchestra.run.smithi052.stdout:generic/242 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:11:06.512 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/242 2024-05-04T14:11:06.512 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/242 2024-05-04T14:11:06.513 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:11:06.516 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:11:06.528 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:11:06.529 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/243) 2024-05-04T14:11:06.789 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:11:06.797 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:11:06.798 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:11:07.300 INFO:teuthology.orchestra.run.smithi052.stdout:generic/243 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:11:07.304 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/243 2024-05-04T14:11:07.304 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/243 2024-05-04T14:11:07.304 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:11:07.307 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:11:07.316 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:11:07.316 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/244) 2024-05-04T14:11:07.578 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:11:07.582 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:11:07.582 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:11:07.769 INFO:teuthology.orchestra.run.smithi052.stdout:generic/244 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-05-04T14:11:07.773 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/244 2024-05-04T14:11:07.773 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/244 2024-05-04T14:11:07.773 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:11:07.776 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:11:07.791 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:11:07.791 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/245) 2024-05-04T14:11:08.024 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:11:08.037 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:11:08.037 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:11:08.713 INFO:teuthology.orchestra.run.smithi052.stdout:generic/245 0s 2024-05-04T14:11:08.806 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/245 2024-05-04T14:11:08.806 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:11:08.809 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:11:08.843 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:11:08.843 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/246) 2024-05-04T14:11:09.360 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:11:09.367 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:11:09.367 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:11:09.943 INFO:teuthology.orchestra.run.smithi052.stdout:generic/246 0s 2024-05-04T14:11:09.952 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/246 2024-05-04T14:11:09.952 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:11:09.954 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:11:09.977 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:11:09.977 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/247) 2024-05-04T14:11:10.527 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:11:10.531 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:11:10.531 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:11:29.000 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:11:29.002 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:11:59.049 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:11:59.053 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:11:59.795 INFO:teuthology.orchestra.run.smithi052.stdout:generic/247 49s 2024-05-04T14:11:59.801 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/247 2024-05-04T14:11:59.801 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:11:59.803 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:11:59.814 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:11:59.814 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/248) 2024-05-04T14:12:00.071 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:00.076 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:00.076 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:00.276 INFO:teuthology.orchestra.run.smithi052.stdout:generic/248 0s 2024-05-04T14:12:00.288 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/248 2024-05-04T14:12:00.288 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:12:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:00.300 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:12:00.300 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/249) 2024-05-04T14:12:00.562 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:00.568 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:02.861 INFO:teuthology.orchestra.run.smithi052.stdout:generic/249 2s 2024-05-04T14:12:02.867 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/249 2024-05-04T14:12:02.867 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:12:02.873 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:02.889 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:12:02.889 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/250) 2024-05-04T14:12:03.480 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:03.494 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:03.494 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:03.662 INFO:teuthology.orchestra.run.smithi052.stdout:generic/250 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:12:03.666 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/250 2024-05-04T14:12:03.666 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/250 2024-05-04T14:12:03.666 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:12:03.670 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:03.680 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:12:03.681 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/251) 2024-05-04T14:12:03.924 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:03.931 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:03.931 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:04.090 INFO:teuthology.orchestra.run.smithi052.stdout:generic/251 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:12:04.100 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/251 2024-05-04T14:12:04.100 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/251 2024-05-04T14:12:04.100 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:12:04.102 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:04.112 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:12:04.112 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/252) 2024-05-04T14:12:04.374 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:04.378 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:04.378 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:04.559 INFO:teuthology.orchestra.run.smithi052.stdout:generic/252 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:12:04.564 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/252 2024-05-04T14:12:04.564 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/252 2024-05-04T14:12:04.564 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:12:04.568 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:04.594 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:12:04.595 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/253) 2024-05-04T14:12:04.809 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:04.814 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:04.814 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:04.964 INFO:teuthology.orchestra.run.smithi052.stdout:generic/253 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:12:04.979 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/253 2024-05-04T14:12:04.979 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/253 2024-05-04T14:12:04.979 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:12:04.982 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:04.999 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:12:04.999 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/254) 2024-05-04T14:12:05.268 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:05.273 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:05.273 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:05.437 INFO:teuthology.orchestra.run.smithi052.stdout:generic/254 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:12:05.441 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/254 2024-05-04T14:12:05.441 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/254 2024-05-04T14:12:05.441 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:12:05.445 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:05.460 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:12:05.460 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/255) 2024-05-04T14:12:05.684 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:05.689 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:05.689 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:07.171 INFO:teuthology.orchestra.run.smithi052.stdout:generic/255 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-04T14:12:07.175 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/255 2024-05-04T14:12:07.175 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/255 2024-05-04T14:12:07.175 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:12:07.178 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:07.193 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:12:07.193 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/256) 2024-05-04T14:12:07.782 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:07.786 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:07.787 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:09.113 INFO:teuthology.orchestra.run.smithi052.stdout:generic/256 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:12:09.117 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/256 2024-05-04T14:12:09.117 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/256 2024-05-04T14:12:09.117 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:12:09.119 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:09.131 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:12:09.131 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/257) 2024-05-04T14:12:09.397 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:09.401 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:09.401 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:11.247 INFO:teuthology.orchestra.run.smithi052.stdout:generic/257 2s 2024-05-04T14:12:11.258 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/257 2024-05-04T14:12:11.259 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:12:11.266 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:11.403 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:12:11.403 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/258) 2024-05-04T14:12:12.538 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:12.544 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:12.747 INFO:teuthology.orchestra.run.smithi052.stdout:generic/258 [failed, exit status 1]- output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/258.out.bad) 2024-05-04T14:12:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/258.out 2024-05-04 12:19:31.976992761 +0000 2024-05-04T14:12:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/258.out.bad 2024-05-04 14:12:12.744602451 +0000 2024-05-04T14:12:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -1,5 +1,6 @@ 2024-05-04T14:12:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: QA output created by 258 2024-05-04T14:12:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: Creating file with timestamp of Jan 1, 1960 2024-05-04T14:12:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: Testing for negative seconds since epoch 2024-05-04T14:12:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: -Remounting to flush cache 2024-05-04T14:12:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: -Testing for negative seconds since epoch 2024-05-04T14:12:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: +Timestamp wrapped: 3979351756 2024-05-04T14:12:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: +Timestamp wrapped 2024-05-04T14:12:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: +(see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/258.full for details) 2024-05-04T14:12:12.767 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/258 2024-05-04T14:12:12.768 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/258 2024-05-04T14:12:12.768 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T14:12:12.770 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:12.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T14:12:12.778 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T14:12:12.778 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T14:12:12.778 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/259) 2024-05-04T14:12:13.039 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:13.045 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:13.200 INFO:teuthology.orchestra.run.smithi052.stdout:generic/259 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:12:13.209 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/259 2024-05-04T14:12:13.209 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/259 2024-05-04T14:12:13.209 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:12:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:13.221 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:12:13.222 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/260) 2024-05-04T14:12:13.464 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:13.468 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:13.619 INFO:teuthology.orchestra.run.smithi052.stdout:generic/260 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:12:13.623 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/260 2024-05-04T14:12:13.623 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/260 2024-05-04T14:12:13.623 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:12:13.625 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:13.636 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:12:13.636 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/261) 2024-05-04T14:12:13.868 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:13.873 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:13.873 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:14.056 INFO:teuthology.orchestra.run.smithi052.stdout:generic/261 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:12:14.060 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/261 2024-05-04T14:12:14.060 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/261 2024-05-04T14:12:14.060 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:12:14.063 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:14.073 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:12:14.074 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/262) 2024-05-04T14:12:14.316 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:14.323 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:14.323 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:14.473 INFO:teuthology.orchestra.run.smithi052.stdout:generic/262 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:12:14.477 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/262 2024-05-04T14:12:14.478 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/262 2024-05-04T14:12:14.478 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:12:14.480 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:14.489 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:12:14.490 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/263) 2024-05-04T14:12:14.730 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:12:14.734 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:12:14.735 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:12:29.131 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:12:29.134 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:12:59.489 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:12:59.491 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:13:29.587 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:13:29.589 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:13:53.257 INFO:teuthology.orchestra.run.smithi052.stdout:generic/263 99s 2024-05-04T14:13:53.265 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/263 2024-05-04T14:13:53.265 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:53.269 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:53.282 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:53.283 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/264) 2024-05-04T14:13:53.817 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:53.821 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:53.821 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:53.961 INFO:teuthology.orchestra.run.smithi052.stdout:generic/264 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:53.965 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/264 2024-05-04T14:13:53.965 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/264 2024-05-04T14:13:53.965 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:53.967 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:53.976 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:53.976 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/265) 2024-05-04T14:13:54.127 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:54.131 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:54.131 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:54.292 INFO:teuthology.orchestra.run.smithi052.stdout:generic/265 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:54.296 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/265 2024-05-04T14:13:54.296 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/265 2024-05-04T14:13:54.296 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:54.298 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:54.306 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:54.307 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/266) 2024-05-04T14:13:54.454 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:54.458 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:54.458 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:54.572 INFO:teuthology.orchestra.run.smithi052.stdout:generic/266 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:54.576 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/266 2024-05-04T14:13:54.576 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/266 2024-05-04T14:13:54.576 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:54.578 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:54.586 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:54.587 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/267) 2024-05-04T14:13:54.736 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:54.740 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:54.740 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:54.868 INFO:teuthology.orchestra.run.smithi052.stdout:generic/267 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:54.872 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/267 2024-05-04T14:13:54.872 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/267 2024-05-04T14:13:54.872 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:54.874 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:54.882 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:54.882 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/268) 2024-05-04T14:13:55.034 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:55.035 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:55.035 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:55.184 INFO:teuthology.orchestra.run.smithi052.stdout:generic/268 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:55.188 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/268 2024-05-04T14:13:55.189 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/268 2024-05-04T14:13:55.189 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:55.191 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:55.198 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:55.199 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/269) 2024-05-04T14:13:55.350 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:55.354 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:55.354 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:55.455 INFO:teuthology.orchestra.run.smithi052.stdout:generic/269 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:55.459 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/269 2024-05-04T14:13:55.459 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/269 2024-05-04T14:13:55.459 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:55.461 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:55.469 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:55.469 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/270) 2024-05-04T14:13:55.620 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:55.624 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:55.624 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:55.740 INFO:teuthology.orchestra.run.smithi052.stdout:generic/270 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-05-04T14:13:55.745 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/270 2024-05-04T14:13:55.745 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/270 2024-05-04T14:13:55.745 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:55.747 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:55.755 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:55.756 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/271) 2024-05-04T14:13:55.902 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:55.905 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:55.906 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:56.032 INFO:teuthology.orchestra.run.smithi052.stdout:generic/271 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:56.036 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/271 2024-05-04T14:13:56.036 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/271 2024-05-04T14:13:56.036 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:56.039 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:56.047 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:56.047 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/272) 2024-05-04T14:13:56.196 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:56.200 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:56.201 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:56.316 INFO:teuthology.orchestra.run.smithi052.stdout:generic/272 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:56.320 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/272 2024-05-04T14:13:56.320 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/272 2024-05-04T14:13:56.320 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:56.322 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:56.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:56.330 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/273) 2024-05-04T14:13:56.478 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:56.482 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:56.482 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:56.583 INFO:teuthology.orchestra.run.smithi052.stdout:generic/273 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:56.587 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/273 2024-05-04T14:13:56.587 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/273 2024-05-04T14:13:56.587 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:56.589 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:56.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:56.597 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/274) 2024-05-04T14:13:56.744 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:56.748 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:56.748 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:56.851 INFO:teuthology.orchestra.run.smithi052.stdout:generic/274 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:56.854 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/274 2024-05-04T14:13:56.855 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/274 2024-05-04T14:13:56.855 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:56.857 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:56.865 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:56.865 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/275) 2024-05-04T14:13:57.012 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:57.017 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:57.017 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:57.116 INFO:teuthology.orchestra.run.smithi052.stdout:generic/275 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:57.120 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/275 2024-05-04T14:13:57.120 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/275 2024-05-04T14:13:57.120 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:57.122 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:57.130 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:57.130 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/276) 2024-05-04T14:13:57.277 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:57.281 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:57.281 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:57.413 INFO:teuthology.orchestra.run.smithi052.stdout:generic/276 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:57.418 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/276 2024-05-04T14:13:57.419 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/276 2024-05-04T14:13:57.419 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:57.421 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:57.430 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:57.430 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/277) 2024-05-04T14:13:57.592 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:57.596 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:57.596 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:57.718 INFO:teuthology.orchestra.run.smithi052.stdout:generic/277 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:57.722 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/277 2024-05-04T14:13:57.722 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/277 2024-05-04T14:13:57.722 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:57.724 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:57.732 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:57.732 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/278) 2024-05-04T14:13:57.887 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:57.891 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:57.892 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:58.017 INFO:teuthology.orchestra.run.smithi052.stdout:generic/278 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:58.021 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/278 2024-05-04T14:13:58.021 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/278 2024-05-04T14:13:58.021 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:58.023 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:58.031 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:58.032 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/279) 2024-05-04T14:13:58.179 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:58.183 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:58.183 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:58.313 INFO:teuthology.orchestra.run.smithi052.stdout:generic/279 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:58.316 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/279 2024-05-04T14:13:58.316 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/279 2024-05-04T14:13:58.316 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:58.319 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:58.327 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:58.327 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/280) 2024-05-04T14:13:58.477 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:58.481 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:58.481 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:58.631 INFO:teuthology.orchestra.run.smithi052.stdout:generic/280 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:58.634 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/280 2024-05-04T14:13:58.635 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/280 2024-05-04T14:13:58.635 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:58.637 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:58.645 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:58.645 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/281) 2024-05-04T14:13:58.795 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:58.799 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:58.799 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:58.924 INFO:teuthology.orchestra.run.smithi052.stdout:generic/281 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:58.928 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/281 2024-05-04T14:13:58.928 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/281 2024-05-04T14:13:58.928 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:58.930 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:58.938 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:58.938 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/282) 2024-05-04T14:13:59.089 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:59.093 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:59.093 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:59.245 INFO:teuthology.orchestra.run.smithi052.stdout:generic/282 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:59.249 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/282 2024-05-04T14:13:59.249 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/282 2024-05-04T14:13:59.249 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:59.251 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:59.259 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:59.260 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/283) 2024-05-04T14:13:59.412 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:59.416 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:59.416 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:59.549 INFO:teuthology.orchestra.run.smithi052.stdout:generic/283 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:59.552 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/283 2024-05-04T14:13:59.552 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/283 2024-05-04T14:13:59.552 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:59.555 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:59.563 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:59.563 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/284) 2024-05-04T14:13:59.624 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:13:59.627 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:13:59.711 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:13:59.715 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:13:59.716 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:59.816 INFO:teuthology.orchestra.run.smithi052.stdout:generic/284 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:13:59.820 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/284 2024-05-04T14:13:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/284 2024-05-04T14:13:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:13:59.822 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:13:59.830 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:13:59.830 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/285) 2024-05-04T14:14:00.024 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:14:00.028 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:14:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:14:02.494 INFO:teuthology.orchestra.run.smithi052.stdout:generic/285 2s 2024-05-04T14:14:02.500 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/285 2024-05-04T14:14:02.500 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:14:02.503 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:14:02.728 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:14:02.728 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/286) 2024-05-04T14:14:03.160 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:14:03.165 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:14:03.165 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:14:29.722 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:14:29.725 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:14:59.877 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:14:59.880 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:15:30.000 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:15:30.003 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:15:32.761 INFO:teuthology.orchestra.run.smithi052.stdout:generic/286 89s 2024-05-04T14:15:32.804 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/286 2024-05-04T14:15:32.804 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:32.807 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:32.821 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:32.821 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/287) 2024-05-04T14:15:33.754 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:33.758 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:33.758 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:33.906 INFO:teuthology.orchestra.run.smithi052.stdout:generic/287 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:33.909 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/287 2024-05-04T14:15:33.909 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/287 2024-05-04T14:15:33.910 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:33.912 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:33.923 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:33.923 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/288) 2024-05-04T14:15:34.076 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:34.081 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:34.081 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:34.185 INFO:teuthology.orchestra.run.smithi052.stdout:generic/288 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:34.189 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/288 2024-05-04T14:15:34.189 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/288 2024-05-04T14:15:34.189 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:34.191 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:34.199 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:34.199 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/289) 2024-05-04T14:15:34.347 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:34.351 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:34.351 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:34.456 INFO:teuthology.orchestra.run.smithi052.stdout:generic/289 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:34.460 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/289 2024-05-04T14:15:34.460 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/289 2024-05-04T14:15:34.460 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:34.462 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:34.470 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:34.470 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/290) 2024-05-04T14:15:34.621 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:34.625 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:34.625 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:34.732 INFO:teuthology.orchestra.run.smithi052.stdout:generic/290 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:34.736 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/290 2024-05-04T14:15:34.736 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/290 2024-05-04T14:15:34.736 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:34.738 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:34.746 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:34.747 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/291) 2024-05-04T14:15:34.900 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:34.904 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:34.904 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:35.009 INFO:teuthology.orchestra.run.smithi052.stdout:generic/291 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:35.013 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/291 2024-05-04T14:15:35.013 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/291 2024-05-04T14:15:35.013 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:35.016 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:35.024 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:35.024 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/292) 2024-05-04T14:15:35.172 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:35.176 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:35.177 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:35.278 INFO:teuthology.orchestra.run.smithi052.stdout:generic/292 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:35.282 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/292 2024-05-04T14:15:35.282 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/292 2024-05-04T14:15:35.282 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:35.284 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:35.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:35.293 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/293) 2024-05-04T14:15:35.444 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:35.448 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:35.448 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:35.551 INFO:teuthology.orchestra.run.smithi052.stdout:generic/293 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:35.554 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/293 2024-05-04T14:15:35.554 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/293 2024-05-04T14:15:35.555 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:35.558 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:35.565 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:35.565 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/294) 2024-05-04T14:15:35.711 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:35.715 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:35.716 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:35.820 INFO:teuthology.orchestra.run.smithi052.stdout:generic/294 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:35.823 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/294 2024-05-04T14:15:35.823 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/294 2024-05-04T14:15:35.824 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:35.826 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:35.834 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:35.835 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/295) 2024-05-04T14:15:35.984 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:35.988 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:35.989 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:36.092 INFO:teuthology.orchestra.run.smithi052.stdout:generic/295 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:36.095 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/295 2024-05-04T14:15:36.095 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/295 2024-05-04T14:15:36.095 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:36.097 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:36.105 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:36.106 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/296) 2024-05-04T14:15:36.255 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:36.259 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:36.259 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:36.361 INFO:teuthology.orchestra.run.smithi052.stdout:generic/296 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:36.364 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/296 2024-05-04T14:15:36.364 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/296 2024-05-04T14:15:36.364 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:36.366 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:36.374 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:36.375 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/297) 2024-05-04T14:15:36.521 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:36.525 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:36.525 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:36.639 INFO:teuthology.orchestra.run.smithi052.stdout:generic/297 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:36.643 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/297 2024-05-04T14:15:36.643 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/297 2024-05-04T14:15:36.643 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:36.645 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:36.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:36.653 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/298) 2024-05-04T14:15:36.801 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:36.805 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:36.805 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:36.917 INFO:teuthology.orchestra.run.smithi052.stdout:generic/298 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:36.921 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/298 2024-05-04T14:15:36.921 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/298 2024-05-04T14:15:36.921 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:36.923 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:36.931 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:36.932 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/299) 2024-05-04T14:15:37.080 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:37.084 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:37.084 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:37.190 INFO:teuthology.orchestra.run.smithi052.stdout:generic/299 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:37.194 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/299 2024-05-04T14:15:37.194 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/299 2024-05-04T14:15:37.194 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:37.196 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:37.204 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:37.204 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/300) 2024-05-04T14:15:37.352 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:37.356 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:37.356 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:37.473 INFO:teuthology.orchestra.run.smithi052.stdout:generic/300 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:37.477 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/300 2024-05-04T14:15:37.477 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/300 2024-05-04T14:15:37.477 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:37.479 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:37.487 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:37.487 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/301) 2024-05-04T14:15:37.635 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:37.639 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:37.639 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:37.741 INFO:teuthology.orchestra.run.smithi052.stdout:generic/301 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:37.745 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/301 2024-05-04T14:15:37.745 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/301 2024-05-04T14:15:37.745 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:37.747 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:37.755 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:37.755 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/302) 2024-05-04T14:15:37.902 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:37.907 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:37.907 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:38.009 INFO:teuthology.orchestra.run.smithi052.stdout:generic/302 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:38.012 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/302 2024-05-04T14:15:38.012 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/302 2024-05-04T14:15:38.012 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:38.014 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:38.023 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:38.023 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/303) 2024-05-04T14:15:38.174 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:38.178 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:38.178 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:38.332 INFO:teuthology.orchestra.run.smithi052.stdout:generic/303 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:15:38.335 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/303 2024-05-04T14:15:38.335 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/303 2024-05-04T14:15:38.335 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:38.337 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:38.345 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:38.345 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/304) 2024-05-04T14:15:38.831 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:38.835 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:38.835 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:38.986 INFO:teuthology.orchestra.run.smithi052.stdout:generic/304 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-05-04T14:15:38.990 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/304 2024-05-04T14:15:38.990 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/304 2024-05-04T14:15:38.990 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:38.992 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:39.000 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:39.000 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/305) 2024-05-04T14:15:39.472 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:39.476 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:39.476 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:39.582 INFO:teuthology.orchestra.run.smithi052.stdout:generic/305 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:39.586 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/305 2024-05-04T14:15:39.586 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/305 2024-05-04T14:15:39.586 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:39.588 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:39.596 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:39.597 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/306) 2024-05-04T14:15:39.743 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:39.747 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:39.748 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:39.849 INFO:teuthology.orchestra.run.smithi052.stdout:generic/306 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:39.853 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/306 2024-05-04T14:15:39.853 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/306 2024-05-04T14:15:39.853 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:39.855 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:39.863 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:39.863 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/307) 2024-05-04T14:15:40.011 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:40.015 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:40.015 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:40.130 INFO:teuthology.orchestra.run.smithi052.stdout:generic/307 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:15:40.134 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/307 2024-05-04T14:15:40.134 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/307 2024-05-04T14:15:40.134 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:40.136 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:40.144 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:40.144 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/308) 2024-05-04T14:15:40.294 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:40.297 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:40.298 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:40.433 INFO:teuthology.orchestra.run.smithi052.stdout:generic/308 0s 2024-05-04T14:15:40.437 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/308 2024-05-04T14:15:40.438 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:40.440 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:40.448 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:40.448 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/309) 2024-05-04T14:15:40.597 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:40.601 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:40.601 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:41.784 INFO:teuthology.orchestra.run.smithi052.stdout:generic/309 1s 2024-05-04T14:15:41.792 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/309 2024-05-04T14:15:41.792 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:15:41.795 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:15:41.808 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:15:41.808 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/310) 2024-05-04T14:15:42.318 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:15:42.321 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:15:42.321 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:16:00.123 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:16:00.126 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:16:30.616 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:16:30.619 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:17:01.208 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:17:01.211 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:17:08.816 INFO:teuthology.orchestra.run.smithi052.stdout:generic/310 86s 2024-05-04T14:17:08.821 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/310 2024-05-04T14:17:08.821 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:17:08.824 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:08.837 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:17:08.838 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/311) 2024-05-04T14:17:09.105 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:17:09.110 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:17:09.110 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:09.661 INFO:teuthology.orchestra.run.smithi052.stdout:generic/311 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:17:09.667 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/311 2024-05-04T14:17:09.667 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/311 2024-05-04T14:17:09.667 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:17:09.669 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:09.704 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:17:09.704 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/312) 2024-05-04T14:17:10.001 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:17:10.005 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:17:10.006 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:10.130 INFO:teuthology.orchestra.run.smithi052.stdout:generic/312 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:17:10.134 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/312 2024-05-04T14:17:10.134 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/312 2024-05-04T14:17:10.134 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:17:10.137 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:10.144 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:17:10.144 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/313) 2024-05-04T14:17:10.288 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:17:10.292 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:17:10.292 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:14.626 INFO:teuthology.orchestra.run.smithi052.stdout:generic/313 4s 2024-05-04T14:17:14.649 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/313 2024-05-04T14:17:14.649 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:17:14.653 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:14.668 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:17:14.668 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/314) 2024-05-04T14:17:14.865 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:17:14.870 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:17:14.870 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:18.682 INFO:teuthology.orchestra.run.smithi052.stdout:generic/314 4s 2024-05-04T14:17:18.698 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/314 2024-05-04T14:17:18.698 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:17:18.701 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:18.714 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:17:18.714 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/315) 2024-05-04T14:17:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:17:18.916 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:17:18.916 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:20.840 INFO:teuthology.orchestra.run.smithi052.stdout:generic/315 2s 2024-05-04T14:17:20.848 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/315 2024-05-04T14:17:20.848 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:17:20.851 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:21.030 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:17:21.031 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/316) 2024-05-04T14:17:21.402 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:17:21.407 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:17:21.407 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:22.478 INFO:teuthology.orchestra.run.smithi052.stdout:generic/316 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-04T14:17:22.482 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/316 2024-05-04T14:17:22.482 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/316 2024-05-04T14:17:22.482 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:17:22.485 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:22.494 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:17:22.494 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/317) 2024-05-04T14:17:23.002 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:17:23.006 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:17:23.006 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:23.132 INFO:teuthology.orchestra.run.smithi052.stdout:generic/317 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:17:23.135 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/317 2024-05-04T14:17:23.135 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/317 2024-05-04T14:17:23.135 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:17:23.138 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:23.146 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:17:23.146 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/318) 2024-05-04T14:17:23.293 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:17:23.297 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:17:23.298 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:23.400 INFO:teuthology.orchestra.run.smithi052.stdout:generic/318 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:17:23.405 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/318 2024-05-04T14:17:23.405 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/318 2024-05-04T14:17:23.405 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:17:23.407 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:23.416 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:17:23.416 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/319) 2024-05-04T14:17:23.561 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:17:23.565 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:17:23.565 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:23.708 INFO:teuthology.orchestra.run.smithi052.stdout:generic/319 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-05-04T14:17:23.712 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/319 2024-05-04T14:17:23.712 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/319 2024-05-04T14:17:23.712 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:17:23.714 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:23.722 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:17:23.722 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/320) 2024-05-04T14:17:23.869 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:17:23.873 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:17:23.873 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:23.994 INFO:teuthology.orchestra.run.smithi052.stdout:generic/320 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:17:23.998 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/320 2024-05-04T14:17:23.998 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/320 2024-05-04T14:17:23.998 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:17:24.000 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:24.008 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:17:24.008 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/321) 2024-05-04T14:17:24.154 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:17:24.158 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:17:24.158 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:24.330 INFO:teuthology.orchestra.run.smithi052.stdout:generic/321 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:17:24.334 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/321 2024-05-04T14:17:24.334 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/321 2024-05-04T14:17:24.334 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:17:24.336 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:24.344 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:17:24.344 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/322) 2024-05-04T14:17:24.492 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:17:24.496 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:17:24.496 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:24.613 INFO:teuthology.orchestra.run.smithi052.stdout:generic/322 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:17:24.617 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/322 2024-05-04T14:17:24.618 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/322 2024-05-04T14:17:24.618 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:17:24.620 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:24.628 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:17:24.628 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/323) 2024-05-04T14:17:24.775 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:17:24.779 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:17:24.779 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:17:31.263 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:17:31.266 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:18:01.338 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:18:01.340 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:18:31.864 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:18:31.866 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:19:01.973 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:19:01.976 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:19:26.424 INFO:teuthology.orchestra.run.smithi052.stdout:generic/323 122s 2024-05-04T14:19:26.430 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/323 2024-05-04T14:19:26.430 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:26.433 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:26.456 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:26.456 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/324) 2024-05-04T14:19:26.979 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:26.984 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:26.984 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:27.114 INFO:teuthology.orchestra.run.smithi052.stdout:generic/324 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:27.118 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/324 2024-05-04T14:19:27.118 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/324 2024-05-04T14:19:27.118 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:27.121 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:27.129 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:27.129 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/325) 2024-05-04T14:19:27.277 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:27.281 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:27.281 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:27.429 INFO:teuthology.orchestra.run.smithi052.stdout:generic/325 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:27.433 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/325 2024-05-04T14:19:27.433 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/325 2024-05-04T14:19:27.433 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:27.435 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:27.443 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:27.443 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/326) 2024-05-04T14:19:27.590 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:27.594 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:27.594 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:27.700 INFO:teuthology.orchestra.run.smithi052.stdout:generic/326 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:27.704 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/326 2024-05-04T14:19:27.704 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/326 2024-05-04T14:19:27.704 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:27.706 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:27.714 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:27.714 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/327) 2024-05-04T14:19:27.859 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:27.863 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:27.863 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:27.966 INFO:teuthology.orchestra.run.smithi052.stdout:generic/327 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:27.970 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/327 2024-05-04T14:19:27.970 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/327 2024-05-04T14:19:27.970 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:27.972 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:27.980 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:27.980 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/328) 2024-05-04T14:19:28.126 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:28.130 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:28.130 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:28.253 INFO:teuthology.orchestra.run.smithi052.stdout:generic/328 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:28.257 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/328 2024-05-04T14:19:28.258 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/328 2024-05-04T14:19:28.258 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:28.260 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:28.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:28.268 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/329) 2024-05-04T14:19:28.415 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:28.419 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:28.553 INFO:teuthology.orchestra.run.smithi052.stdout:generic/329 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:28.557 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/329 2024-05-04T14:19:28.557 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/329 2024-05-04T14:19:28.557 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:28.560 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:28.569 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:28.569 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/330) 2024-05-04T14:19:28.719 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:28.723 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:28.723 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:28.825 INFO:teuthology.orchestra.run.smithi052.stdout:generic/330 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:28.829 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/330 2024-05-04T14:19:28.829 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/330 2024-05-04T14:19:28.829 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:28.831 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:28.839 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:28.839 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/331) 2024-05-04T14:19:28.985 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:28.989 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:28.989 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:29.113 INFO:teuthology.orchestra.run.smithi052.stdout:generic/331 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:29.117 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/331 2024-05-04T14:19:29.117 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/331 2024-05-04T14:19:29.117 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:29.119 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:29.127 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:29.127 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/332) 2024-05-04T14:19:29.277 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:29.281 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:29.281 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:29.386 INFO:teuthology.orchestra.run.smithi052.stdout:generic/332 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:29.390 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/332 2024-05-04T14:19:29.390 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/332 2024-05-04T14:19:29.390 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:29.392 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:29.400 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:29.400 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/333) 2024-05-04T14:19:29.548 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:29.552 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:29.552 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:29.657 INFO:teuthology.orchestra.run.smithi052.stdout:generic/333 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:29.660 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/333 2024-05-04T14:19:29.660 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/333 2024-05-04T14:19:29.660 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:29.663 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:29.671 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:29.671 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/334) 2024-05-04T14:19:29.821 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:29.825 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:29.825 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:29.935 INFO:teuthology.orchestra.run.smithi052.stdout:generic/334 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:29.939 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/334 2024-05-04T14:19:29.939 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/334 2024-05-04T14:19:29.939 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:29.941 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:29.949 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:29.949 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/335) 2024-05-04T14:19:30.096 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:30.100 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:30.101 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:30.222 INFO:teuthology.orchestra.run.smithi052.stdout:generic/335 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:30.225 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/335 2024-05-04T14:19:30.225 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/335 2024-05-04T14:19:30.225 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:30.228 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:30.235 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:30.236 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/336) 2024-05-04T14:19:30.382 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:30.386 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:30.386 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:30.502 INFO:teuthology.orchestra.run.smithi052.stdout:generic/336 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:30.506 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/336 2024-05-04T14:19:30.506 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/336 2024-05-04T14:19:30.506 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:30.508 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:30.516 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:30.516 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/337) 2024-05-04T14:19:30.662 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:30.667 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:30.667 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:30.770 INFO:teuthology.orchestra.run.smithi052.stdout:generic/337 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:30.774 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/337 2024-05-04T14:19:30.774 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/337 2024-05-04T14:19:30.774 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:30.776 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:30.784 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:30.784 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/338) 2024-05-04T14:19:30.934 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:30.938 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:30.938 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:31.053 INFO:teuthology.orchestra.run.smithi052.stdout:generic/338 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:31.056 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/338 2024-05-04T14:19:31.057 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/338 2024-05-04T14:19:31.057 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:31.059 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:31.067 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:31.067 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/339) 2024-05-04T14:19:31.215 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:31.219 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:31.219 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:31.320 INFO:teuthology.orchestra.run.smithi052.stdout:generic/339 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:31.324 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/339 2024-05-04T14:19:31.324 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/339 2024-05-04T14:19:31.324 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:31.326 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:31.334 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:31.334 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/340) 2024-05-04T14:19:31.481 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:31.485 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:31.485 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:31.585 INFO:teuthology.orchestra.run.smithi052.stdout:generic/340 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:31.588 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/340 2024-05-04T14:19:31.588 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/340 2024-05-04T14:19:31.588 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:31.590 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:31.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:31.598 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/341) 2024-05-04T14:19:31.745 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:31.749 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:31.749 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:31.874 INFO:teuthology.orchestra.run.smithi052.stdout:generic/341 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:31.878 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/341 2024-05-04T14:19:31.878 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/341 2024-05-04T14:19:31.878 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:31.880 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:31.890 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:31.890 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/342) 2024-05-04T14:19:32.038 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:32.042 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:32.042 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:32.088 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:19:32.090 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:19:32.154 INFO:teuthology.orchestra.run.smithi052.stdout:generic/342 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:32.158 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/342 2024-05-04T14:19:32.158 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/342 2024-05-04T14:19:32.158 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:32.160 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:32.168 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:32.168 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/343) 2024-05-04T14:19:32.361 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:32.365 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:32.365 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:32.490 INFO:teuthology.orchestra.run.smithi052.stdout:generic/343 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:32.493 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/343 2024-05-04T14:19:32.493 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/343 2024-05-04T14:19:32.493 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:32.496 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:32.503 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:32.504 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/344) 2024-05-04T14:19:32.649 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:32.653 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:32.653 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:32.755 INFO:teuthology.orchestra.run.smithi052.stdout:generic/344 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:32.758 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/344 2024-05-04T14:19:32.758 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/344 2024-05-04T14:19:32.759 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:32.760 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:32.768 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:32.768 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/345) 2024-05-04T14:19:32.914 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:32.918 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:32.918 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:33.017 INFO:teuthology.orchestra.run.smithi052.stdout:generic/345 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:33.021 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/345 2024-05-04T14:19:33.021 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/345 2024-05-04T14:19:33.021 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:33.023 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:33.031 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:33.031 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/346) 2024-05-04T14:19:33.178 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:33.182 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:33.182 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:33.282 INFO:teuthology.orchestra.run.smithi052.stdout:generic/346 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:33.286 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/346 2024-05-04T14:19:33.286 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/346 2024-05-04T14:19:33.286 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:33.288 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:33.296 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:33.296 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/347) 2024-05-04T14:19:33.447 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:33.447 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:33.448 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:33.614 INFO:teuthology.orchestra.run.smithi052.stdout:generic/347 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:33.617 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/347 2024-05-04T14:19:33.617 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/347 2024-05-04T14:19:33.618 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:33.620 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:33.627 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:33.628 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/348) 2024-05-04T14:19:33.777 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:33.781 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:33.781 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:33.899 INFO:teuthology.orchestra.run.smithi052.stdout:generic/348 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:33.902 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/348 2024-05-04T14:19:33.902 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/348 2024-05-04T14:19:33.902 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:33.905 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:33.913 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:33.914 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/349) 2024-05-04T14:19:34.064 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:34.068 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:34.068 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:35.417 INFO:teuthology.orchestra.run.smithi052.stdout:generic/349 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-05-04T14:19:35.423 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/349 2024-05-04T14:19:35.423 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/349 2024-05-04T14:19:35.423 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:35.427 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:35.440 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:35.440 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/350) 2024-05-04T14:19:35.632 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:35.637 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:35.638 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:40.440 INFO:teuthology.orchestra.run.smithi052.stdout:generic/350 5s 2024-05-04T14:19:40.489 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/350 2024-05-04T14:19:40.490 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:40.495 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:40.508 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:40.508 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/351) 2024-05-04T14:19:40.712 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:40.716 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:40.717 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:43.296 INFO:teuthology.orchestra.run.smithi052.stdout:generic/351 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-05-04T14:19:43.301 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/351 2024-05-04T14:19:43.301 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/351 2024-05-04T14:19:43.302 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:43.305 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:43.317 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:43.317 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/352) 2024-05-04T14:19:43.866 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:43.871 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:43.871 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:44.216 INFO:teuthology.orchestra.run.smithi052.stdout:generic/352 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:44.222 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/352 2024-05-04T14:19:44.222 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/352 2024-05-04T14:19:44.222 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:44.226 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:44.238 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:44.239 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/353) 2024-05-04T14:19:44.445 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:44.450 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:44.450 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:44.591 INFO:teuthology.orchestra.run.smithi052.stdout:generic/353 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:44.596 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/353 2024-05-04T14:19:44.596 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/353 2024-05-04T14:19:44.596 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:44.599 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:44.609 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:44.610 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/354) 2024-05-04T14:19:44.795 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:44.800 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:44.800 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:44.925 INFO:teuthology.orchestra.run.smithi052.stdout:generic/354 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:44.929 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/354 2024-05-04T14:19:44.929 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/354 2024-05-04T14:19:44.930 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:44.932 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:44.943 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:44.943 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/355) 2024-05-04T14:19:45.123 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:45.127 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:45.128 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:generic/355 0s 2024-05-04T14:19:45.793 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/355 2024-05-04T14:19:45.793 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:45.796 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:45.808 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:45.808 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/356) 2024-05-04T14:19:46.309 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:46.313 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:46.313 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:46.425 INFO:teuthology.orchestra.run.smithi052.stdout:generic/356 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:46.428 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/356 2024-05-04T14:19:46.429 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/356 2024-05-04T14:19:46.429 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:46.431 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:46.439 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:46.439 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/357) 2024-05-04T14:19:46.585 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:46.589 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:46.589 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:46.696 INFO:teuthology.orchestra.run.smithi052.stdout:generic/357 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:46.699 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/357 2024-05-04T14:19:46.699 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/357 2024-05-04T14:19:46.699 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:46.702 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:46.710 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:46.710 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/358) 2024-05-04T14:19:46.865 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:46.869 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:46.869 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:46.991 INFO:teuthology.orchestra.run.smithi052.stdout:generic/358 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:46.995 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/358 2024-05-04T14:19:46.995 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/358 2024-05-04T14:19:46.995 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:46.998 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:47.005 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:47.005 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/359) 2024-05-04T14:19:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:47.159 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:47.159 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:47.261 INFO:teuthology.orchestra.run.smithi052.stdout:generic/359 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:47.265 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/359 2024-05-04T14:19:47.265 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/359 2024-05-04T14:19:47.265 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:47.267 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:47.275 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:47.275 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/360) 2024-05-04T14:19:47.423 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:47.427 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:47.428 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:47.615 INFO:teuthology.orchestra.run.smithi052.stdout:generic/360 0s 2024-05-04T14:19:47.621 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/360 2024-05-04T14:19:47.621 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:47.624 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:47.633 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:47.633 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/361) 2024-05-04T14:19:47.783 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:47.787 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:47.787 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:47.942 INFO:teuthology.orchestra.run.smithi052.stdout:generic/361 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:47.946 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/361 2024-05-04T14:19:47.946 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/361 2024-05-04T14:19:47.946 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:47.949 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:47.957 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:47.958 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/362) 2024-05-04T14:19:48.106 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:48.110 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:48.110 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:48.212 INFO:teuthology.orchestra.run.smithi052.stdout:generic/362 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:48.216 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/362 2024-05-04T14:19:48.217 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/362 2024-05-04T14:19:48.217 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:48.219 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:48.227 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:48.227 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/363) 2024-05-04T14:19:48.375 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:48.379 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:48.379 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:48.481 INFO:teuthology.orchestra.run.smithi052.stdout:generic/363 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:48.485 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/363 2024-05-04T14:19:48.485 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/363 2024-05-04T14:19:48.485 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:48.487 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:48.495 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:48.495 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/364) 2024-05-04T14:19:48.641 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:48.645 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:48.645 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:48.788 INFO:teuthology.orchestra.run.smithi052.stdout:generic/364 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:48.793 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/364 2024-05-04T14:19:48.794 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/364 2024-05-04T14:19:48.794 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:48.797 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:48.805 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:48.805 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/365) 2024-05-04T14:19:48.956 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:48.960 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:48.960 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:49.059 INFO:teuthology.orchestra.run.smithi052.stdout:generic/365 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:49.063 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/365 2024-05-04T14:19:49.063 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/365 2024-05-04T14:19:49.063 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:49.066 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:49.074 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:49.074 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/366) 2024-05-04T14:19:49.227 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:49.231 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:49.231 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:49.331 INFO:teuthology.orchestra.run.smithi052.stdout:generic/366 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:49.336 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/366 2024-05-04T14:19:49.336 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/366 2024-05-04T14:19:49.336 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:49.338 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:49.345 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:49.346 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/367) 2024-05-04T14:19:49.494 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:49.498 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:49.498 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:49.601 INFO:teuthology.orchestra.run.smithi052.stdout:generic/367 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:49.605 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/367 2024-05-04T14:19:49.605 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/367 2024-05-04T14:19:49.605 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:49.607 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:49.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:49.615 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/368) 2024-05-04T14:19:49.760 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:49.764 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:49.764 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:49.868 INFO:teuthology.orchestra.run.smithi052.stdout:generic/368 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:49.871 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/368 2024-05-04T14:19:49.871 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/368 2024-05-04T14:19:49.872 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:49.874 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:49.881 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:49.882 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/369) 2024-05-04T14:19:50.028 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:50.033 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:50.033 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:50.132 INFO:teuthology.orchestra.run.smithi052.stdout:generic/369 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:50.136 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/369 2024-05-04T14:19:50.136 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/369 2024-05-04T14:19:50.136 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:50.138 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:50.146 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:50.146 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/370) 2024-05-04T14:19:50.292 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:50.297 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:50.297 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:50.399 INFO:teuthology.orchestra.run.smithi052.stdout:generic/370 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:50.402 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/370 2024-05-04T14:19:50.402 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/370 2024-05-04T14:19:50.403 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:50.405 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:50.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:50.413 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/371) 2024-05-04T14:19:50.560 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:50.564 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:50.564 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:50.668 INFO:teuthology.orchestra.run.smithi052.stdout:generic/371 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:50.672 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/371 2024-05-04T14:19:50.672 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/371 2024-05-04T14:19:50.672 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:50.674 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:50.682 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:50.683 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/372) 2024-05-04T14:19:50.828 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:50.832 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:50.832 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:50.935 INFO:teuthology.orchestra.run.smithi052.stdout:generic/372 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:50.938 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/372 2024-05-04T14:19:50.938 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/372 2024-05-04T14:19:50.939 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:50.941 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:50.949 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:50.949 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/373) 2024-05-04T14:19:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:51.099 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:51.099 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:51.204 INFO:teuthology.orchestra.run.smithi052.stdout:generic/373 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:51.208 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/373 2024-05-04T14:19:51.208 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/373 2024-05-04T14:19:51.208 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:51.210 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:51.218 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:51.218 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/374) 2024-05-04T14:19:51.365 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:51.369 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:51.369 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:51.471 INFO:teuthology.orchestra.run.smithi052.stdout:generic/374 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:51.475 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/374 2024-05-04T14:19:51.475 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/374 2024-05-04T14:19:51.475 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:51.477 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:51.485 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:51.485 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/375) 2024-05-04T14:19:51.631 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:51.635 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:51.636 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:51.773 INFO:teuthology.orchestra.run.smithi052.stdout:generic/375 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-05-04T14:19:51.777 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/375 2024-05-04T14:19:51.777 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/375 2024-05-04T14:19:51.777 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:51.779 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:51.787 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:51.787 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/376) 2024-05-04T14:19:51.936 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:51.940 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:51.940 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:52.070 INFO:teuthology.orchestra.run.smithi052.stdout:generic/376 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:52.074 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/376 2024-05-04T14:19:52.074 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/376 2024-05-04T14:19:52.074 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:52.076 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:52.084 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:52.085 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/377) 2024-05-04T14:19:52.233 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:52.237 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:52.237 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:52.354 INFO:teuthology.orchestra.run.smithi052.stdout:generic/377 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:52.358 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/377 2024-05-04T14:19:52.358 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/377 2024-05-04T14:19:52.358 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:52.361 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:52.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:52.369 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/378) 2024-05-04T14:19:52.516 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:52.520 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:52.520 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:52.708 INFO:teuthology.orchestra.run.smithi052.stdout:generic/378 0s 2024-05-04T14:19:52.714 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/378 2024-05-04T14:19:52.714 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:52.716 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:52.724 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:52.724 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/379) 2024-05-04T14:19:53.223 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:53.227 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:53.227 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:53.343 INFO:teuthology.orchestra.run.smithi052.stdout:generic/379 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:53.347 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/379 2024-05-04T14:19:53.347 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/379 2024-05-04T14:19:53.347 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:53.349 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:53.356 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:53.357 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/380) 2024-05-04T14:19:53.505 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:53.509 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:53.509 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:53.612 INFO:teuthology.orchestra.run.smithi052.stdout:generic/380 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:53.616 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/380 2024-05-04T14:19:53.616 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/380 2024-05-04T14:19:53.616 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:53.618 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:53.626 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:53.627 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/381) 2024-05-04T14:19:53.772 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:53.776 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:53.776 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:53.880 INFO:teuthology.orchestra.run.smithi052.stdout:generic/381 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:53.884 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/381 2024-05-04T14:19:53.884 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/381 2024-05-04T14:19:53.884 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:53.886 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:53.894 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:53.894 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/382) 2024-05-04T14:19:54.040 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:54.044 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:54.044 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:54.149 INFO:teuthology.orchestra.run.smithi052.stdout:generic/382 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:54.152 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/382 2024-05-04T14:19:54.152 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/382 2024-05-04T14:19:54.152 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:54.154 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:54.162 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:54.162 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/383) 2024-05-04T14:19:54.308 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:54.312 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:54.312 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:54.415 INFO:teuthology.orchestra.run.smithi052.stdout:generic/383 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:54.418 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/383 2024-05-04T14:19:54.418 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/383 2024-05-04T14:19:54.418 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:54.420 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:54.428 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:54.428 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/384) 2024-05-04T14:19:54.574 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:54.578 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:54.578 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:54.689 INFO:teuthology.orchestra.run.smithi052.stdout:generic/384 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-05-04T14:19:54.693 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/384 2024-05-04T14:19:54.693 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/384 2024-05-04T14:19:54.693 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:54.695 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:54.704 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:54.704 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/385) 2024-05-04T14:19:54.852 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:54.856 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:54.856 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:54.964 INFO:teuthology.orchestra.run.smithi052.stdout:generic/385 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:54.967 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/385 2024-05-04T14:19:54.968 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/385 2024-05-04T14:19:54.968 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:54.970 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:54.978 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:54.978 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/386) 2024-05-04T14:19:55.127 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:55.131 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:55.131 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:55.234 INFO:teuthology.orchestra.run.smithi052.stdout:generic/386 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-05-04T14:19:55.238 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/386 2024-05-04T14:19:55.238 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/386 2024-05-04T14:19:55.238 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:55.240 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:55.248 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:55.248 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/387) 2024-05-04T14:19:55.394 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:55.399 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:55.399 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:55.501 INFO:teuthology.orchestra.run.smithi052.stdout:generic/387 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:55.505 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/387 2024-05-04T14:19:55.505 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/387 2024-05-04T14:19:55.505 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:55.507 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:55.514 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:55.515 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/388) 2024-05-04T14:19:55.661 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:55.665 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:55.665 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:55.776 INFO:teuthology.orchestra.run.smithi052.stdout:generic/388 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:55.779 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/388 2024-05-04T14:19:55.780 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/388 2024-05-04T14:19:55.780 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:55.782 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:55.790 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:55.791 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/389) 2024-05-04T14:19:55.938 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:55.943 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:55.943 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:56.063 INFO:teuthology.orchestra.run.smithi052.stdout:generic/389 [not run] O_TMPFILE is not supported 2024-05-04T14:19:56.067 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/389 2024-05-04T14:19:56.067 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/389 2024-05-04T14:19:56.067 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:56.069 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:56.077 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:56.077 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/390) 2024-05-04T14:19:56.227 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:56.231 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:56.231 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:56.368 INFO:teuthology.orchestra.run.smithi052.stdout:generic/390 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:19:56.371 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/390 2024-05-04T14:19:56.372 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/390 2024-05-04T14:19:56.372 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:19:56.374 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:19:56.382 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:19:56.382 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/391) 2024-05-04T14:19:56.530 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:19:56.534 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:19:56.534 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:02.195 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:20:02.198 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:20:32.316 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:20:32.319 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:20:38.643 INFO:teuthology.orchestra.run.smithi052.stdout:generic/391 42s 2024-05-04T14:20:38.792 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/391 2024-05-04T14:20:38.793 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:38.797 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:38.827 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:38.827 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/392) 2024-05-04T14:20:39.940 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:39.945 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:40.165 INFO:teuthology.orchestra.run.smithi052.stdout:generic/392 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:40.169 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/392 2024-05-04T14:20:40.169 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/392 2024-05-04T14:20:40.169 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:40.171 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:40.181 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:40.181 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/393) 2024-05-04T14:20:40.396 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:40.401 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:40.401 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:40.567 INFO:teuthology.orchestra.run.smithi052.stdout:generic/393 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:40.571 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/393 2024-05-04T14:20:40.571 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/393 2024-05-04T14:20:40.571 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:40.574 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:40.589 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:40.589 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/394) 2024-05-04T14:20:40.835 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:40.840 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:40.840 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:41.072 INFO:teuthology.orchestra.run.smithi052.stdout:generic/394 1s 2024-05-04T14:20:41.083 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/394 2024-05-04T14:20:41.083 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:41.086 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:41.100 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:41.100 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/395) 2024-05-04T14:20:41.640 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:41.645 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:41.645 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:41.853 INFO:teuthology.orchestra.run.smithi052.stdout:generic/395 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:41.857 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/395 2024-05-04T14:20:41.857 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/395 2024-05-04T14:20:41.857 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:41.860 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:41.879 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:41.879 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/396) 2024-05-04T14:20:42.113 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:42.119 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:42.119 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:42.325 INFO:teuthology.orchestra.run.smithi052.stdout:generic/396 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:42.329 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/396 2024-05-04T14:20:42.329 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/396 2024-05-04T14:20:42.329 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:42.331 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:42.346 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:42.347 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/397) 2024-05-04T14:20:42.639 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:42.644 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:42.644 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:42.858 INFO:teuthology.orchestra.run.smithi052.stdout:generic/397 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:42.863 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/397 2024-05-04T14:20:42.863 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/397 2024-05-04T14:20:42.863 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:42.866 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:42.875 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:42.875 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/398) 2024-05-04T14:20:43.120 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:43.130 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:43.130 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:43.338 INFO:teuthology.orchestra.run.smithi052.stdout:generic/398 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:43.345 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/398 2024-05-04T14:20:43.346 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/398 2024-05-04T14:20:43.346 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:43.348 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:43.359 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:43.359 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/399) 2024-05-04T14:20:43.604 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:43.610 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:43.611 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:43.817 INFO:teuthology.orchestra.run.smithi052.stdout:generic/399 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:43.822 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/399 2024-05-04T14:20:43.822 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/399 2024-05-04T14:20:43.822 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:43.825 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:43.841 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:43.841 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/400) 2024-05-04T14:20:44.073 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:44.078 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:44.078 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:44.239 INFO:teuthology.orchestra.run.smithi052.stdout:generic/400 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-05-04T14:20:44.242 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/400 2024-05-04T14:20:44.242 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/400 2024-05-04T14:20:44.242 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:44.245 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:44.257 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:44.258 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/401) 2024-05-04T14:20:44.485 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:44.493 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:44.493 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:44.635 INFO:teuthology.orchestra.run.smithi052.stdout:generic/401 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:44.640 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/401 2024-05-04T14:20:44.640 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/401 2024-05-04T14:20:44.640 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:44.642 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:44.652 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:44.652 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/402) 2024-05-04T14:20:44.884 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:44.888 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:44.888 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:45.014 INFO:teuthology.orchestra.run.smithi052.stdout:generic/402 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:45.017 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/402 2024-05-04T14:20:45.017 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/402 2024-05-04T14:20:45.017 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:45.019 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:45.028 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:45.028 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/403) 2024-05-04T14:20:45.174 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:45.178 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:45.178 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:45.279 INFO:teuthology.orchestra.run.smithi052.stdout:generic/403 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:45.282 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/403 2024-05-04T14:20:45.283 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/403 2024-05-04T14:20:45.283 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:45.285 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:45.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:45.293 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/404) 2024-05-04T14:20:45.439 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:45.443 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:45.443 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:47.204 INFO:teuthology.orchestra.run.smithi052.stdout:generic/404 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-05-04T14:20:47.209 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/404 2024-05-04T14:20:47.209 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/404 2024-05-04T14:20:47.209 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:47.213 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:47.224 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:47.224 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/405) 2024-05-04T14:20:47.772 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:47.777 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:47.777 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:48.025 INFO:teuthology.orchestra.run.smithi052.stdout:generic/405 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:48.031 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/405 2024-05-04T14:20:48.032 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/405 2024-05-04T14:20:48.032 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:48.035 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:48.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:48.046 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/406) 2024-05-04T14:20:48.235 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:48.240 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:48.240 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:48.356 INFO:teuthology.orchestra.run.smithi052.stdout:generic/406 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:48.360 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/406 2024-05-04T14:20:48.360 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/406 2024-05-04T14:20:48.360 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:48.362 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:48.371 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:48.371 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/407) 2024-05-04T14:20:48.523 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:48.527 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:48.527 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:48.682 INFO:teuthology.orchestra.run.smithi052.stdout:generic/407 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:20:48.685 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/407 2024-05-04T14:20:48.686 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/407 2024-05-04T14:20:48.686 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:48.688 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:48.697 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:48.697 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/408) 2024-05-04T14:20:49.183 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:49.187 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:49.187 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:49.346 INFO:teuthology.orchestra.run.smithi052.stdout:generic/408 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-05-04T14:20:49.350 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/408 2024-05-04T14:20:49.350 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/408 2024-05-04T14:20:49.350 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:49.352 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:49.361 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:49.361 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/409) 2024-05-04T14:20:49.862 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:49.866 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:49.866 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:49.972 INFO:teuthology.orchestra.run.smithi052.stdout:generic/409 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:49.975 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/409 2024-05-04T14:20:49.975 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/409 2024-05-04T14:20:49.975 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:49.977 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:49.986 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:49.986 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/410) 2024-05-04T14:20:50.133 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:50.137 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:50.137 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:50.243 INFO:teuthology.orchestra.run.smithi052.stdout:generic/410 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:50.246 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/410 2024-05-04T14:20:50.247 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/410 2024-05-04T14:20:50.247 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:50.249 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:50.257 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:50.257 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/411) 2024-05-04T14:20:50.402 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:50.407 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:50.407 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:50.512 INFO:teuthology.orchestra.run.smithi052.stdout:generic/411 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:50.516 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/411 2024-05-04T14:20:50.516 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/411 2024-05-04T14:20:50.516 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:50.518 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:50.526 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:50.527 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/412) 2024-05-04T14:20:50.676 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:50.681 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:50.681 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:50.783 INFO:teuthology.orchestra.run.smithi052.stdout:generic/412 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:50.787 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/412 2024-05-04T14:20:50.787 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/412 2024-05-04T14:20:50.787 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:50.789 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:50.797 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:50.797 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/413) 2024-05-04T14:20:50.945 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:50.948 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:50.949 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:51.055 INFO:teuthology.orchestra.run.smithi052.stdout:generic/413 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:51.058 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/413 2024-05-04T14:20:51.058 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/413 2024-05-04T14:20:51.058 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:51.060 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:51.069 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:51.069 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/414) 2024-05-04T14:20:51.218 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:51.222 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:51.222 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:51.324 INFO:teuthology.orchestra.run.smithi052.stdout:generic/414 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:51.328 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/414 2024-05-04T14:20:51.328 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/414 2024-05-04T14:20:51.328 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:51.330 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:51.338 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:51.338 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/415) 2024-05-04T14:20:51.486 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:51.490 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:51.490 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:51.590 INFO:teuthology.orchestra.run.smithi052.stdout:generic/415 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:51.594 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/415 2024-05-04T14:20:51.594 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/415 2024-05-04T14:20:51.594 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:51.596 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:51.604 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:51.605 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/416) 2024-05-04T14:20:51.750 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:51.755 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:51.755 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:generic/416 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/416 2024-05-04T14:20:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/416 2024-05-04T14:20:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:51.862 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:51.871 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:51.871 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/417) 2024-05-04T14:20:52.017 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:52.021 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:52.021 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:52.120 INFO:teuthology.orchestra.run.smithi052.stdout:generic/417 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:52.124 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/417 2024-05-04T14:20:52.124 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/417 2024-05-04T14:20:52.124 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:52.126 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:52.134 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:52.134 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/418) 2024-05-04T14:20:52.283 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:52.287 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:52.287 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:52.434 INFO:teuthology.orchestra.run.smithi052.stdout:generic/418 [not run] require ceph-fuse to be valid block disk 2024-05-04T14:20:52.437 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/418 2024-05-04T14:20:52.438 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/418 2024-05-04T14:20:52.438 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:52.440 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:52.448 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:52.448 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/419) 2024-05-04T14:20:52.597 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:52.601 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:52.601 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:52.728 INFO:teuthology.orchestra.run.smithi052.stdout:generic/419 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:52.732 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/419 2024-05-04T14:20:52.732 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/419 2024-05-04T14:20:52.732 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:52.734 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:52.742 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:52.743 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/420) 2024-05-04T14:20:52.890 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:52.894 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:52.894 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:54.025 INFO:teuthology.orchestra.run.smithi052.stdout:generic/420 2s 2024-05-04T14:20:54.033 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/420 2024-05-04T14:20:54.033 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:54.036 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:54.144 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:54.145 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/421) 2024-05-04T14:20:54.590 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:54.594 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:54.594 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:54.726 INFO:teuthology.orchestra.run.smithi052.stdout:generic/421 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:54.729 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/421 2024-05-04T14:20:54.730 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/421 2024-05-04T14:20:54.730 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:54.732 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:54.740 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:54.740 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/422) 2024-05-04T14:20:54.891 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:54.895 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:54.895 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:55.001 INFO:teuthology.orchestra.run.smithi052.stdout:generic/422 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:55.005 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/422 2024-05-04T14:20:55.005 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/422 2024-05-04T14:20:55.005 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:55.007 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:55.015 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:55.015 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/423) 2024-05-04T14:20:55.161 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:55.165 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:55.165 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:55.396 INFO:teuthology.orchestra.run.smithi052.stdout:generic/423 0s 2024-05-04T14:20:55.401 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/423 2024-05-04T14:20:55.401 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:55.404 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:55.412 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:55.412 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/424) 2024-05-04T14:20:55.898 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:55.902 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:55.902 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:56.076 INFO:teuthology.orchestra.run.smithi052.stdout:generic/424 [not run] file system doesn't support any of /usr/bin/chattr +a/+c/+d/+i 2024-05-04T14:20:56.080 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/424 2024-05-04T14:20:56.080 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/424 2024-05-04T14:20:56.080 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:56.082 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:56.091 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:56.092 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/425) 2024-05-04T14:20:56.241 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:56.245 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:56.245 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:56.348 INFO:teuthology.orchestra.run.smithi052.stdout:generic/425 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:20:56.351 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/425 2024-05-04T14:20:56.351 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/425 2024-05-04T14:20:56.351 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:20:56.353 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:20:56.361 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:20:56.362 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/426) 2024-05-04T14:20:56.510 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:20:56.514 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:20:56.514 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:03.978 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:21:03.981 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:21:34.094 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:21:34.097 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:21:50.361 INFO:teuthology.orchestra.run.smithi052.stdout:generic/426 54s 2024-05-04T14:21:50.512 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/426 2024-05-04T14:21:50.512 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:21:50.517 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:50.629 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:21:50.629 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/427) 2024-05-04T14:21:52.105 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:21:52.111 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:21:52.111 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:52.324 INFO:teuthology.orchestra.run.smithi052.stdout:generic/427 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:21:52.329 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/427 2024-05-04T14:21:52.329 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/427 2024-05-04T14:21:52.329 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:21:52.333 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:52.344 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:21:52.344 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/428) 2024-05-04T14:21:52.541 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:21:52.546 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:21:52.547 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:52.738 INFO:teuthology.orchestra.run.smithi052.stdout:generic/428 0s 2024-05-04T14:21:52.745 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/428 2024-05-04T14:21:52.745 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:21:52.748 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:52.757 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:21:52.758 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/429) 2024-05-04T14:21:52.957 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:21:52.961 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:21:52.961 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:53.138 INFO:teuthology.orchestra.run.smithi052.stdout:generic/429 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:21:53.141 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/429 2024-05-04T14:21:53.142 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/429 2024-05-04T14:21:53.142 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:21:53.144 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:53.153 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:21:53.153 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/430) 2024-05-04T14:21:53.314 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:21:53.318 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:21:53.318 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:53.595 INFO:teuthology.orchestra.run.smithi052.stdout:generic/430 0s 2024-05-04T14:21:53.600 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/430 2024-05-04T14:21:53.600 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:21:53.603 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:53.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:21:53.611 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/431) 2024-05-04T14:21:54.088 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:21:54.092 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:21:54.093 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:54.334 INFO:teuthology.orchestra.run.smithi052.stdout:generic/431 0s 2024-05-04T14:21:54.339 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/431 2024-05-04T14:21:54.339 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:21:54.341 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:54.350 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:21:54.350 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/432) 2024-05-04T14:21:54.838 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:21:54.841 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:21:54.841 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:55.077 INFO:teuthology.orchestra.run.smithi052.stdout:generic/432 1s 2024-05-04T14:21:55.082 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/432 2024-05-04T14:21:55.082 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:21:55.084 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:55.092 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:21:55.092 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/433) 2024-05-04T14:21:55.576 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:21:55.580 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:21:55.581 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:55.801 INFO:teuthology.orchestra.run.smithi052.stdout:generic/433 0s 2024-05-04T14:21:55.808 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/433 2024-05-04T14:21:55.808 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:21:55.811 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:55.818 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:21:55.818 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/434) 2024-05-04T14:21:56.318 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:21:56.322 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:21:56.322 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:56.682 INFO:teuthology.orchestra.run.smithi052.stdout:generic/434 - output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/434.out.bad) 2024-05-04T14:21:56.685 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/434.out 2024-05-04 12:19:31.988992582 +0000 2024-05-04T14:21:56.685 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/434.out.bad 2024-05-04 14:21:56.679822546 +0000 2024-05-04T14:21:56.685 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -9,6 +9,6 @@ 2024-05-04T14:21:56.685 INFO:teuthology.orchestra.run.smithi052.stdout: copy_range: Bad file descriptor 2024-05-04T14:21:56.685 INFO:teuthology.orchestra.run.smithi052.stdout: d41d8cd98f00b204e9800998ecf8427e TEST_DIR/test-434/copy 2024-05-04T14:21:56.685 INFO:teuthology.orchestra.run.smithi052.stdout: Try to copy to a device 2024-05-04T14:21:56.685 INFO:teuthology.orchestra.run.smithi052.stdout: -copy_range: Invalid argument 2024-05-04T14:21:56.685 INFO:teuthology.orchestra.run.smithi052.stdout: +/tmp/tmp.FGpMFsgr0Qtest/test-434/dev1: Permission denied 2024-05-04T14:21:56.686 INFO:teuthology.orchestra.run.smithi052.stdout: Try to copy to a fifo 2024-05-04T14:21:56.686 INFO:teuthology.orchestra.run.smithi052.stdout: copy_range: Invalid argument 2024-05-04T14:21:56.687 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/434 2024-05-04T14:21:56.687 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/434 2024-05-04T14:21:56.687 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T14:21:56.690 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:56.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T14:21:56.700 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T14:21:56.700 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T14:21:56.700 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/435) 2024-05-04T14:21:57.188 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:21:57.192 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:21:57.192 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:57.332 INFO:teuthology.orchestra.run.smithi052.stdout:generic/435 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:21:57.336 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/435 2024-05-04T14:21:57.336 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/435 2024-05-04T14:21:57.336 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:21:57.338 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:57.346 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:21:57.346 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/436) 2024-05-04T14:21:57.502 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:21:57.505 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:21:57.505 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:59.289 INFO:teuthology.orchestra.run.smithi052.stdout:generic/436 2s 2024-05-04T14:21:59.297 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/436 2024-05-04T14:21:59.297 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:21:59.300 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:21:59.410 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:21:59.410 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/437) 2024-05-04T14:21:59.878 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:21:59.882 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:21:59.882 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:22:00.784 INFO:teuthology.orchestra.run.smithi052.stdout:generic/437 1s 2024-05-04T14:22:00.792 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/437 2024-05-04T14:22:00.792 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:22:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:22:00.850 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:22:00.851 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/438) 2024-05-04T14:22:01.058 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:22:01.063 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:22:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:22:05.503 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:22:05.505 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:22:35.622 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:22:35.625 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:23:05.937 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:23:05.939 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:23:08.950 INFO:teuthology.orchestra.run.smithi052.stdout:generic/438 67s 2024-05-04T14:23:08.955 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/438 2024-05-04T14:23:08.955 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:08.957 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:09.016 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:09.016 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/439) 2024-05-04T14:23:09.177 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:09.181 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:09.182 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:09.288 INFO:teuthology.orchestra.run.smithi052.stdout:generic/439 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:09.292 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/439 2024-05-04T14:23:09.292 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/439 2024-05-04T14:23:09.292 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:09.294 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:09.302 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:09.303 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/440) 2024-05-04T14:23:09.458 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:09.462 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:09.462 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:09.591 INFO:teuthology.orchestra.run.smithi052.stdout:generic/440 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:09.595 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/440 2024-05-04T14:23:09.596 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/440 2024-05-04T14:23:09.596 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:09.597 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:09.605 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:09.605 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/441) 2024-05-04T14:23:09.761 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:09.765 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:09.765 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:09.927 INFO:teuthology.orchestra.run.smithi052.stdout:generic/441 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:09.932 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/441 2024-05-04T14:23:09.932 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/441 2024-05-04T14:23:09.932 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:09.934 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:09.943 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:09.943 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/442) 2024-05-04T14:23:10.106 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:10.110 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:10.226 INFO:teuthology.orchestra.run.smithi052.stdout:generic/442 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:10.230 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/442 2024-05-04T14:23:10.230 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/442 2024-05-04T14:23:10.230 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:10.232 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:10.240 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:10.241 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/443) 2024-05-04T14:23:10.398 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:10.403 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:10.403 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:10.549 INFO:teuthology.orchestra.run.smithi052.stdout:generic/443 0s 2024-05-04T14:23:10.554 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/443 2024-05-04T14:23:10.554 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:10.556 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:10.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:10.564 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/444) 2024-05-04T14:23:10.722 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:10.726 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:10.726 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:10.866 INFO:teuthology.orchestra.run.smithi052.stdout:generic/444 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-05-04T14:23:10.870 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/444 2024-05-04T14:23:10.870 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/444 2024-05-04T14:23:10.870 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:10.872 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:10.880 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:10.880 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/445) 2024-05-04T14:23:11.043 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:11.047 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:11.047 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:13.119 INFO:teuthology.orchestra.run.smithi052.stdout:generic/445 2s 2024-05-04T14:23:13.127 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/445 2024-05-04T14:23:13.128 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:13.143 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:13.144 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/446) 2024-05-04T14:23:13.704 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:13.709 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:13.709 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:13.825 INFO:teuthology.orchestra.run.smithi052.stdout:generic/446 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:13.830 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/446 2024-05-04T14:23:13.830 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/446 2024-05-04T14:23:13.830 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:13.833 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:13.842 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:13.842 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/447) 2024-05-04T14:23:14.030 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:14.035 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:14.035 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:14.156 INFO:teuthology.orchestra.run.smithi052.stdout:generic/447 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:14.160 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/447 2024-05-04T14:23:14.160 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/447 2024-05-04T14:23:14.160 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:14.162 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:14.171 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:14.171 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/448) 2024-05-04T14:23:14.328 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:14.332 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:14.332 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:16.053 INFO:teuthology.orchestra.run.smithi052.stdout:generic/448 2s 2024-05-04T14:23:16.060 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/448 2024-05-04T14:23:16.060 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:16.064 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:16.247 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:16.247 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/449) 2024-05-04T14:23:16.634 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:16.638 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:16.638 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:16.758 INFO:teuthology.orchestra.run.smithi052.stdout:generic/449 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:16.762 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/449 2024-05-04T14:23:16.762 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/449 2024-05-04T14:23:16.762 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:16.765 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:16.775 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:16.775 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/450) 2024-05-04T14:23:16.953 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:16.958 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:16.958 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:18.352 INFO:teuthology.orchestra.run.smithi052.stdout:generic/450 2s 2024-05-04T14:23:18.360 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/450 2024-05-04T14:23:18.360 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:18.364 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:18.376 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:18.376 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/451) 2024-05-04T14:23:18.900 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:18.904 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:18.904 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:19.322 INFO:teuthology.orchestra.run.smithi052.stdout:generic/451 - output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/451.out.bad) 2024-05-04T14:23:19.324 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/451.out 2024-05-04 12:19:31.988992582 +0000 2024-05-04T14:23:19.324 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/451.out.bad 2024-05-04 14:23:19.318522047 +0000 2024-05-04T14:23:19.324 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -1,2 +1,9 @@ 2024-05-04T14:23:19.324 INFO:teuthology.orchestra.run.smithi052.stdout: QA output created by 451 2024-05-04T14:23:19.324 INFO:teuthology.orchestra.run.smithi052.stdout: +get stale data from buffer read 2024-05-04T14:23:19.324 INFO:teuthology.orchestra.run.smithi052.stdout: +00000000 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 UUUUUUUUUUUUUUUU 2024-05-04T14:23:19.324 INFO:teuthology.orchestra.run.smithi052.stdout: +* 2024-05-04T14:23:19.325 INFO:teuthology.orchestra.run.smithi052.stdout: +00080000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 2024-05-04T14:23:19.325 INFO:teuthology.orchestra.run.smithi052.stdout: +* 2024-05-04T14:23:19.325 INFO:teuthology.orchestra.run.smithi052.stdout: +0008c000 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 UUUUUUUUUUUUUUUU 2024-05-04T14:23:19.325 INFO:teuthology.orchestra.run.smithi052.stdout: +000a0000 2024-05-04T14:23:19.325 INFO:teuthology.orchestra.run.smithi052.stdout: Silence is golden 2024-05-04T14:23:19.326 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/451 2024-05-04T14:23:19.326 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/451 2024-05-04T14:23:19.326 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T14:23:19.328 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:19.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T14:23:19.390 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T14:23:19.391 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T14:23:19.391 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/452) 2024-05-04T14:23:19.844 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:19.862 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:19.862 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:19.977 INFO:teuthology.orchestra.run.smithi052.stdout:generic/452 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:19.981 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/452 2024-05-04T14:23:19.981 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/452 2024-05-04T14:23:19.981 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:19.983 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:19.991 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:19.991 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/453) 2024-05-04T14:23:20.135 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:20.158 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:20.158 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:20.239 INFO:teuthology.orchestra.run.smithi052.stdout:generic/453 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/453 2024-05-04T14:23:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/453 2024-05-04T14:23:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:20.245 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:20.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:20.253 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/454) 2024-05-04T14:23:20.404 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:20.408 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:20.408 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:20.513 INFO:teuthology.orchestra.run.smithi052.stdout:generic/454 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:20.516 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/454 2024-05-04T14:23:20.516 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/454 2024-05-04T14:23:20.516 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:20.518 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:20.526 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:20.527 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/455) 2024-05-04T14:23:20.673 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:20.677 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:20.677 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:20.875 INFO:teuthology.orchestra.run.smithi052.stdout:generic/455 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:20.881 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/455 2024-05-04T14:23:20.881 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/455 2024-05-04T14:23:20.881 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:20.883 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:20.892 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:20.892 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/456) 2024-05-04T14:23:21.038 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:21.042 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:21.043 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:21.157 INFO:teuthology.orchestra.run.smithi052.stdout:generic/456 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:21.160 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/456 2024-05-04T14:23:21.160 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/456 2024-05-04T14:23:21.160 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:21.163 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:21.171 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:21.171 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/457) 2024-05-04T14:23:21.320 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:21.324 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:21.324 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:21.532 INFO:teuthology.orchestra.run.smithi052.stdout:generic/457 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:21.536 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/457 2024-05-04T14:23:21.536 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/457 2024-05-04T14:23:21.536 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:21.539 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:21.547 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:21.547 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/458) 2024-05-04T14:23:21.695 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:21.699 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:21.699 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:generic/458 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/458 2024-05-04T14:23:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/458 2024-05-04T14:23:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:21.807 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:21.815 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:21.815 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/459) 2024-05-04T14:23:21.965 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:21.970 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:21.970 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:22.239 INFO:teuthology.orchestra.run.smithi052.stdout:generic/459 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:22.244 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/459 2024-05-04T14:23:22.245 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/459 2024-05-04T14:23:22.245 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:22.249 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:22.260 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:22.260 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/460) 2024-05-04T14:23:22.417 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:22.422 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:22.422 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:22.526 INFO:teuthology.orchestra.run.smithi052.stdout:generic/460 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:22.530 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/460 2024-05-04T14:23:22.530 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/460 2024-05-04T14:23:22.530 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:22.533 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:22.540 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:22.541 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/461) 2024-05-04T14:23:22.690 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:22.694 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:22.694 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:22.799 INFO:teuthology.orchestra.run.smithi052.stdout:generic/461 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:22.802 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/461 2024-05-04T14:23:22.803 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/461 2024-05-04T14:23:22.803 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:22.805 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:22.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:22.813 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/462) 2024-05-04T14:23:22.960 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:22.964 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:22.965 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:23.073 INFO:teuthology.orchestra.run.smithi052.stdout:generic/462 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:23.077 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/462 2024-05-04T14:23:23.077 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/462 2024-05-04T14:23:23.077 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:23.079 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:23.087 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:23.087 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/463) 2024-05-04T14:23:23.235 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:23.239 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:23.239 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:23.399 INFO:teuthology.orchestra.run.smithi052.stdout:generic/463 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T14:23:23.403 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/463 2024-05-04T14:23:23.403 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/463 2024-05-04T14:23:23.403 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:23.405 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:23.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:23.414 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/464) 2024-05-04T14:23:23.908 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:23.912 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:23.912 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:24.012 INFO:teuthology.orchestra.run.smithi052.stdout:generic/464 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:24.015 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/464 2024-05-04T14:23:24.015 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/464 2024-05-04T14:23:24.015 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:24.018 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:24.026 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:24.026 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/465) 2024-05-04T14:23:24.175 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:24.180 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:24.180 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:35.983 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:23:35.986 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:23:46.987 INFO:teuthology.orchestra.run.smithi052.stdout:generic/465 22s 2024-05-04T14:23:47.109 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/465 2024-05-04T14:23:47.109 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:47.113 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:47.125 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:47.125 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/466) 2024-05-04T14:23:47.430 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:47.434 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:47.434 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:47.538 INFO:teuthology.orchestra.run.smithi052.stdout:generic/466 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:47.542 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/466 2024-05-04T14:23:47.542 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/466 2024-05-04T14:23:47.542 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:47.544 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:47.552 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:47.552 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/467) 2024-05-04T14:23:47.720 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:47.724 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:47.724 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:55.400 INFO:teuthology.orchestra.run.smithi052.stdout:generic/467 - output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/467.out.bad) 2024-05-04T14:23:55.415 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/467.out 2024-05-04 12:19:31.992992523 +0000 2024-05-04T14:23:55.415 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/467.out.bad 2024-05-04 14:23:55.389973036 +0000 2024-05-04T14:23:55.415 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -1,5 +1,15 @@ 2024-05-04T14:23:55.415 INFO:teuthology.orchestra.run.smithi052.stdout: QA output created by 467 2024-05-04T14:23:55.415 INFO:teuthology.orchestra.run.smithi052.stdout: test_file_handles TEST_DIR/467-dir -dp 2024-05-04T14:23:55.415 INFO:teuthology.orchestra.run.smithi052.stdout: +open_by_handle(/tmp/tmp.FGpMFsgr0Qtest/467-dir/file000000) opened an unlinked file! 2024-05-04T14:23:55.415 INFO:teuthology.orchestra.run.smithi052.stdout: +open_by_handle(/tmp/tmp.FGpMFsgr0Qtest/467-dir/file000001) opened an unlinked file! 2024-05-04T14:23:55.415 INFO:teuthology.orchestra.run.smithi052.stdout: +open_by_handle(/tmp/tmp.FGpMFsgr0Qtest/467-dir/file000002) opened an unlinked file! 2024-05-04T14:23:55.416 INFO:teuthology.orchestra.run.smithi052.stdout: +open_by_handle(/tmp/tmp.FGpMFsgr0Qtest/467-dir/file000003) opened an unlinked file! 2024-05-04T14:23:55.416 INFO:teuthology.orchestra.run.smithi052.stdout: +open_by_handle(/tmp/tmp.FGpMFsgr0Qtest/467-dir/file000004) opened an unlinked file! 2024-05-04T14:23:55.416 INFO:teuthology.orchestra.run.smithi052.stdout: +open_by_handle(/tmp/tmp.FGpMFsgr0Qtest/467-dir/file000005) opened an unlinked file! 2024-05-04T14:23:55.416 INFO:teuthology.orchestra.run.smithi052.stdout: +open_by_handle(/tmp/tmp.FGpMFsgr0Qtest/467-dir/file000006) opened an unlinked file! 2024-05-04T14:23:55.416 INFO:teuthology.orchestra.run.smithi052.stdout: +open_by_handle(/tmp/tmp.FGpMFsgr0Qtest/467-dir/file000007) opened an unlinked file! 2024-05-04T14:23:55.416 INFO:teuthology.orchestra.run.smithi052.stdout: +open_by_handle(/tmp/tmp.FGpMFsgr0Qtest/467-dir/file000008) opened an unlinked file! 2024-05-04T14:23:55.416 INFO:teuthology.orchestra.run.smithi052.stdout: +open_by_handle(/tmp/tmp.FGpMFsgr0Qtest/467-dir/file000009) opened an unlinked file! 2024-05-04T14:23:55.416 INFO:teuthology.orchestra.run.smithi052.stdout: test_file_handles TEST_DIR/467-dir -rp 2024-05-04T14:23:55.416 INFO:teuthology.orchestra.run.smithi052.stdout: test_file_handles TEST_DIR/467-dir -dkr 2024-05-04T14:23:55.416 INFO:teuthology.orchestra.run.smithi052.stdout: test_file_handles TEST_DIR/467-dir -lr 2024-05-04T14:23:55.433 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/467 2024-05-04T14:23:55.433 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/467 2024-05-04T14:23:55.433 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T14:23:55.470 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:55.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T14:23:55.561 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T14:23:55.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T14:23:55.561 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/468) 2024-05-04T14:23:57.130 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:57.136 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:57.136 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:57.335 INFO:teuthology.orchestra.run.smithi052.stdout:generic/468 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:23:57.340 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/468 2024-05-04T14:23:57.340 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/468 2024-05-04T14:23:57.340 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:23:57.344 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:23:57.354 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:23:57.354 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/469) 2024-05-04T14:23:57.564 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:23:57.569 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:23:57.569 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:24:01.073 INFO:teuthology.orchestra.run.smithi052.stdout:generic/469 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-05-04T14:24:01.079 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/469 2024-05-04T14:24:01.079 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/469 2024-05-04T14:24:01.079 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:24:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:24:01.095 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:24:01.096 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/470) 2024-05-04T14:24:01.622 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:24:01.627 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:24:01.627 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:24:01.785 INFO:teuthology.orchestra.run.smithi052.stdout:generic/470 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:24:01.789 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/470 2024-05-04T14:24:01.790 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/470 2024-05-04T14:24:01.790 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:24:01.792 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:24:01.802 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:24:01.802 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/471) 2024-05-04T14:24:01.969 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:24:01.973 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:24:01.973 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:24:06.140 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:24:06.143 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:24:36.233 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:24:36.235 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:25:06.343 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:25:06.345 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:25:06.377 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-04T14:25:06.372+0000 7f6f26d7f640 -1 Fail to open '/proc/356772/cmdline' error = (2) No such file or directory 2024-05-04T14:25:06.377 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-04T14:25:06.372+0000 7f6f26d7f640 -1 received signal: Hangup from (PID: 356772) UID: 0 2024-05-04T14:25:06.377 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T14:25:06.372+0000 7fbf91e1b640 -1 Fail to open '/proc/356772/cmdline' error = (2) No such file or directory 2024-05-04T14:25:06.377 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T14:25:06.372+0000 7fbf91e1b640 -1 received signal: Hangup from (PID: 356772) UID: 0 2024-05-04T14:25:31.423 INFO:teuthology.orchestra.run.smithi052.stdout:generic/471 90s 2024-05-04T14:25:31.439 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/471 2024-05-04T14:25:31.440 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:25:31.445 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:25:31.457 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:25:31.458 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/472) 2024-05-04T14:25:32.284 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:25:32.288 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:25:32.288 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:25:32.430 INFO:teuthology.orchestra.run.smithi052.stdout:generic/472 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:25:32.450 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/472 2024-05-04T14:25:32.451 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/472 2024-05-04T14:25:32.451 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:25:32.455 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:25:32.493 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:25:32.493 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/473) 2024-05-04T14:25:33.279 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:25:33.285 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:25:33.285 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:25:33.493 INFO:teuthology.orchestra.run.smithi052.stdout:generic/473 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-04T14:25:33.496 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/473 2024-05-04T14:25:33.496 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/473 2024-05-04T14:25:33.496 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:25:33.498 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:25:33.507 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:25:33.507 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/474) 2024-05-04T14:25:33.656 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:25:33.660 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:25:33.660 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:25:33.781 INFO:teuthology.orchestra.run.smithi052.stdout:generic/474 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:25:33.784 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/474 2024-05-04T14:25:33.784 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/474 2024-05-04T14:25:33.784 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:25:33.787 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:25:33.795 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:25:33.796 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/475) 2024-05-04T14:25:33.944 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:25:33.948 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:25:33.948 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:25:34.075 INFO:teuthology.orchestra.run.smithi052.stdout:generic/475 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:25:34.078 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/475 2024-05-04T14:25:34.079 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/475 2024-05-04T14:25:34.079 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:25:34.081 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:25:34.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:25:34.089 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/476) 2024-05-04T14:25:34.237 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:25:34.241 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:25:34.241 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:25:34.351 INFO:teuthology.orchestra.run.smithi052.stdout:generic/476 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:25:34.354 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/476 2024-05-04T14:25:34.355 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/476 2024-05-04T14:25:34.355 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:25:34.357 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:25:34.365 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:25:34.365 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/477) 2024-05-04T14:25:34.515 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:25:34.519 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:25:34.519 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:25:56.395 INFO:teuthology.orchestra.run.smithi052.stdout:generic/477 22s 2024-05-04T14:25:56.810 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/477 2024-05-04T14:25:56.810 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:25:56.817 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:25:56.911 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:25:56.911 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/478) 2024-05-04T14:26:00.448 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:00.455 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:11.177 INFO:teuthology.orchestra.run.smithi052.stdout:generic/478 - output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/478.out.bad) 2024-05-04T14:26:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/478.out 2024-05-04 12:19:31.992992523 +0000 2024-05-04T14:26:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/478.out.bad 2024-05-04 14:26:11.155992944 +0000 2024-05-04T14:26:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -10,7 +10,7 @@ 2024-05-04T14:26:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: get wrlck 2024-05-04T14:26:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: -lock could be placed 2024-05-04T14:26:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: +get wrlck 2024-05-04T14:26:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -25,19 +25,19 @@ 2024-05-04T14:26:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: get wrlck 2024-05-04T14:26:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: -lock could be placed 2024-05-04T14:26:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: +get wrlck 2024-05-04T14:26:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: get rdlck 2024-05-04T14:26:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: -lock could be placed 2024-05-04T14:26:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: +get rdlck 2024-05-04T14:26:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: get rdlck 2024-05-04T14:26:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: get rdlck 2024-05-04T14:26:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: get rdlck 2024-05-04T14:26:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: get rdlck 2024-05-04T14:26:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: -lock could be placed 2024-05-04T14:26:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: +get rdlck 2024-05-04T14:26:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: get rdlck 2024-05-04T14:26:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: get rdlck 2024-05-04T14:26:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -46,7 +46,7 @@ 2024-05-04T14:26:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: get rdlck 2024-05-04T14:26:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: get rdlck 2024-05-04T14:26:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: -lock could be placed 2024-05-04T14:26:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: +get rdlck 2024-05-04T14:26:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: lock could be placed 2024-05-04T14:26:11.183 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/478 2024-05-04T14:26:11.183 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/478 2024-05-04T14:26:11.184 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T14:26:11.186 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:11.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T14:26:11.237 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T14:26:11.237 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T14:26:11.238 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/479) 2024-05-04T14:26:11.412 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:11.416 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:11.416 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:11.721 INFO:teuthology.orchestra.run.smithi052.stdout:generic/479 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:11.727 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/479 2024-05-04T14:26:11.727 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/479 2024-05-04T14:26:11.727 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:11.731 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:11.742 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:11.742 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/480) 2024-05-04T14:26:11.921 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:11.925 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:11.925 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:12.048 INFO:teuthology.orchestra.run.smithi052.stdout:generic/480 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:12.052 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/480 2024-05-04T14:26:12.052 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/480 2024-05-04T14:26:12.052 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:12.054 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:12.062 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:12.063 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/481) 2024-05-04T14:26:12.226 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:12.230 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:12.230 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:12.348 INFO:teuthology.orchestra.run.smithi052.stdout:generic/481 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:12.352 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/481 2024-05-04T14:26:12.352 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/481 2024-05-04T14:26:12.352 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:12.354 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:12.363 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:12.363 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/482) 2024-05-04T14:26:12.517 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:12.522 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:12.725 INFO:teuthology.orchestra.run.smithi052.stdout:generic/482 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:12.730 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/482 2024-05-04T14:26:12.730 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/482 2024-05-04T14:26:12.730 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:12.740 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:12.741 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/483) 2024-05-04T14:26:12.890 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:12.894 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:13.028 INFO:teuthology.orchestra.run.smithi052.stdout:generic/483 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:13.031 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/483 2024-05-04T14:26:13.032 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/483 2024-05-04T14:26:13.032 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:13.042 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:13.043 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/484) 2024-05-04T14:26:13.205 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:13.209 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:13.347 INFO:teuthology.orchestra.run.smithi052.stdout:generic/484 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:13.351 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/484 2024-05-04T14:26:13.351 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/484 2024-05-04T14:26:13.351 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:13.362 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:13.362 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/485) 2024-05-04T14:26:13.512 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:13.516 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:14.526 INFO:teuthology.orchestra.run.smithi052.stdout:generic/485 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-05-04T14:26:14.530 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/485 2024-05-04T14:26:14.530 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/485 2024-05-04T14:26:14.530 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:14.532 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:14.541 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:14.542 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/486) 2024-05-04T14:26:15.079 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:15.083 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:15.084 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:15.244 INFO:teuthology.orchestra.run.smithi052.stdout:generic/486 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:15.248 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/486 2024-05-04T14:26:15.248 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/486 2024-05-04T14:26:15.248 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:15.250 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:15.259 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:15.259 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/487) 2024-05-04T14:26:15.408 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:15.412 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:15.412 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:15.537 INFO:teuthology.orchestra.run.smithi052.stdout:generic/487 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:15.540 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/487 2024-05-04T14:26:15.541 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/487 2024-05-04T14:26:15.541 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:15.543 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:15.552 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:15.552 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/488) 2024-05-04T14:26:15.704 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:15.708 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:15.812 INFO:teuthology.orchestra.run.smithi052.stdout:generic/488 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:15.815 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/488 2024-05-04T14:26:15.816 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/488 2024-05-04T14:26:15.816 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:15.818 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:15.826 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:15.826 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/489) 2024-05-04T14:26:15.978 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:15.982 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:15.982 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:16.104 INFO:teuthology.orchestra.run.smithi052.stdout:generic/489 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:16.107 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/489 2024-05-04T14:26:16.107 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/489 2024-05-04T14:26:16.107 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:16.110 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:16.118 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:16.118 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/490) 2024-05-04T14:26:16.267 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:16.271 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:16.271 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:17.578 INFO:teuthology.orchestra.run.smithi052.stdout:generic/490 1s 2024-05-04T14:26:17.583 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/490 2024-05-04T14:26:17.583 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:17.586 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:17.815 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:17.815 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/491) 2024-05-04T14:26:18.161 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:18.166 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:18.166 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:18.293 INFO:teuthology.orchestra.run.smithi052.stdout:generic/491 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:18.296 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/491 2024-05-04T14:26:18.296 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/491 2024-05-04T14:26:18.297 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:18.299 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:18.307 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:18.307 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/492) 2024-05-04T14:26:18.477 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:18.481 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:18.481 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:18.583 INFO:teuthology.orchestra.run.smithi052.stdout:generic/492 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:18.586 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/492 2024-05-04T14:26:18.586 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/492 2024-05-04T14:26:18.587 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:18.589 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:18.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:18.598 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/493) 2024-05-04T14:26:18.747 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:18.751 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:18.751 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:18.865 INFO:teuthology.orchestra.run.smithi052.stdout:generic/493 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:18.868 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/493 2024-05-04T14:26:18.868 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/493 2024-05-04T14:26:18.868 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:18.871 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:18.879 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:18.879 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/494) 2024-05-04T14:26:19.040 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:19.044 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:19.044 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:19.147 INFO:teuthology.orchestra.run.smithi052.stdout:generic/494 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:19.150 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/494 2024-05-04T14:26:19.150 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/494 2024-05-04T14:26:19.150 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:19.152 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:19.160 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:19.160 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/495) 2024-05-04T14:26:19.310 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:19.314 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:19.315 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:19.417 INFO:teuthology.orchestra.run.smithi052.stdout:generic/495 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:19.420 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/495 2024-05-04T14:26:19.421 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/495 2024-05-04T14:26:19.421 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:19.423 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:19.432 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:19.432 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/496) 2024-05-04T14:26:19.587 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:19.591 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:19.591 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:19.708 INFO:teuthology.orchestra.run.smithi052.stdout:generic/496 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:19.711 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/496 2024-05-04T14:26:19.711 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/496 2024-05-04T14:26:19.711 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:19.714 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:19.722 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:19.723 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/497) 2024-05-04T14:26:19.876 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:19.882 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:19.882 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:19.982 INFO:teuthology.orchestra.run.smithi052.stdout:generic/497 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/497 2024-05-04T14:26:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/497 2024-05-04T14:26:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:19.988 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:19.996 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:19.997 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/498) 2024-05-04T14:26:20.146 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:20.150 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:20.150 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:generic/498 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:20.271 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/498 2024-05-04T14:26:20.271 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/498 2024-05-04T14:26:20.271 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:20.273 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:20.281 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:20.282 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/499) 2024-05-04T14:26:20.430 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:20.434 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:20.434 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:20.544 INFO:teuthology.orchestra.run.smithi052.stdout:generic/499 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:20.548 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/499 2024-05-04T14:26:20.548 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/499 2024-05-04T14:26:20.548 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:20.550 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:20.559 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:20.559 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/500) 2024-05-04T14:26:20.796 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:20.800 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:20.800 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:20.975 INFO:teuthology.orchestra.run.smithi052.stdout:generic/500 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:20.978 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/500 2024-05-04T14:26:20.979 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/500 2024-05-04T14:26:20.979 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:20.981 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:20.988 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:20.989 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/501) 2024-05-04T14:26:21.140 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:21.143 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:21.143 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:21.260 INFO:teuthology.orchestra.run.smithi052.stdout:generic/501 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:21.263 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/501 2024-05-04T14:26:21.263 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/501 2024-05-04T14:26:21.264 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:21.266 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:21.273 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:21.274 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/502) 2024-05-04T14:26:21.423 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:21.427 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:21.427 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:21.544 INFO:teuthology.orchestra.run.smithi052.stdout:generic/502 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:21.548 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/502 2024-05-04T14:26:21.548 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/502 2024-05-04T14:26:21.548 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:21.550 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:21.558 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:21.558 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/503) 2024-05-04T14:26:21.710 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:21.714 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:21.714 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:21.822 INFO:teuthology.orchestra.run.smithi052.stdout:generic/503 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/503 2024-05-04T14:26:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/503 2024-05-04T14:26:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:21.827 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:21.836 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:21.836 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/504) 2024-05-04T14:26:21.986 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:21.990 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:21.990 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:22.192 INFO:teuthology.orchestra.run.smithi052.stdout:generic/504 - output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/504.out.bad) 2024-05-04T14:26:22.193 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/504.out 2024-05-04 12:19:31.992992523 +0000 2024-05-04T14:26:22.194 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/504.out.bad 2024-05-04 14:26:22.187837359 +0000 2024-05-04T14:26:22.194 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -1,2 +1,3 @@ 2024-05-04T14:26:22.194 INFO:teuthology.orchestra.run.smithi052.stdout: QA output created by 504 2024-05-04T14:26:22.194 INFO:teuthology.orchestra.run.smithi052.stdout: +lock info not found 2024-05-04T14:26:22.194 INFO:teuthology.orchestra.run.smithi052.stdout: Silence is golden 2024-05-04T14:26:22.196 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/504 2024-05-04T14:26:22.196 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/504 2024-05-04T14:26:22.196 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T14:26:22.198 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:22.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T14:26:22.207 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T14:26:22.207 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T14:26:22.207 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/505) 2024-05-04T14:26:22.356 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:22.360 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:22.361 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:22.464 INFO:teuthology.orchestra.run.smithi052.stdout:generic/505 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:22.468 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/505 2024-05-04T14:26:22.468 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/505 2024-05-04T14:26:22.468 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:22.470 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:22.478 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:22.479 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/506) 2024-05-04T14:26:22.626 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:22.630 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:22.631 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:22.758 INFO:teuthology.orchestra.run.smithi052.stdout:generic/506 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:22.761 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/506 2024-05-04T14:26:22.762 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/506 2024-05-04T14:26:22.762 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:22.764 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:22.772 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:22.773 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/507) 2024-05-04T14:26:22.919 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:22.923 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:22.923 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:23.103 INFO:teuthology.orchestra.run.smithi052.stdout:generic/507 [not run] file system doesn't support chattr +ASai 2024-05-04T14:26:23.107 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/507 2024-05-04T14:26:23.107 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/507 2024-05-04T14:26:23.107 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:23.110 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:23.119 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:23.119 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/508) 2024-05-04T14:26:23.266 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:23.270 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:23.270 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:23.381 INFO:teuthology.orchestra.run.smithi052.stdout:generic/508 [not run] lsattr not supported by test filesystem type: ceph-fuse 2024-05-04T14:26:23.385 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/508 2024-05-04T14:26:23.385 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/508 2024-05-04T14:26:23.385 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:23.387 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:23.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:23.395 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/509) 2024-05-04T14:26:23.545 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:23.549 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:23.549 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:23.664 INFO:teuthology.orchestra.run.smithi052.stdout:generic/509 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:23.668 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/509 2024-05-04T14:26:23.668 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/509 2024-05-04T14:26:23.668 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:23.670 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:23.679 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:23.679 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/510) 2024-05-04T14:26:23.827 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:23.832 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:23.832 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:23.952 INFO:teuthology.orchestra.run.smithi052.stdout:generic/510 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:23.955 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/510 2024-05-04T14:26:23.955 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/510 2024-05-04T14:26:23.955 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:23.958 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:23.966 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:23.967 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/511) 2024-05-04T14:26:24.118 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:24.123 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:24.123 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:24.233 INFO:teuthology.orchestra.run.smithi052.stdout:generic/511 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:24.237 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/511 2024-05-04T14:26:24.237 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/511 2024-05-04T14:26:24.237 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:24.239 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:24.248 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:24.248 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/512) 2024-05-04T14:26:24.398 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:24.402 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:24.402 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:24.528 INFO:teuthology.orchestra.run.smithi052.stdout:generic/512 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:24.531 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/512 2024-05-04T14:26:24.532 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/512 2024-05-04T14:26:24.532 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:24.534 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:24.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:24.545 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/513) 2024-05-04T14:26:24.701 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:24.705 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:24.706 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:24.808 INFO:teuthology.orchestra.run.smithi052.stdout:generic/513 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/513 2024-05-04T14:26:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/513 2024-05-04T14:26:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:24.814 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:24.822 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:24.823 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/514) 2024-05-04T14:26:24.978 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:24.983 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:24.983 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:25.086 INFO:teuthology.orchestra.run.smithi052.stdout:generic/514 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:25.090 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/514 2024-05-04T14:26:25.091 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/514 2024-05-04T14:26:25.091 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:25.093 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:25.102 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:25.102 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/515) 2024-05-04T14:26:25.253 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:25.257 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:25.257 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:25.362 INFO:teuthology.orchestra.run.smithi052.stdout:generic/515 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:25.365 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/515 2024-05-04T14:26:25.365 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/515 2024-05-04T14:26:25.365 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:25.368 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:25.376 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:25.376 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/516) 2024-05-04T14:26:25.524 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:25.528 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:25.528 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:25.688 INFO:teuthology.orchestra.run.smithi052.stdout:generic/516 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-05-04T14:26:25.692 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/516 2024-05-04T14:26:25.692 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/516 2024-05-04T14:26:25.692 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:25.694 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:25.703 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:25.703 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/517) 2024-05-04T14:26:26.211 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:26.215 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:26.215 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:26.320 INFO:teuthology.orchestra.run.smithi052.stdout:generic/517 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:26.323 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/517 2024-05-04T14:26:26.323 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/517 2024-05-04T14:26:26.323 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:26.327 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:26.334 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:26.334 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/518) 2024-05-04T14:26:26.483 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:26.487 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:26.487 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:26.596 INFO:teuthology.orchestra.run.smithi052.stdout:generic/518 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:26.600 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/518 2024-05-04T14:26:26.600 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/518 2024-05-04T14:26:26.600 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:26.603 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:26.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:26.611 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/519) 2024-05-04T14:26:26.767 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:26.771 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:26.873 INFO:teuthology.orchestra.run.smithi052.stdout:generic/519 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:26.877 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/519 2024-05-04T14:26:26.877 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/519 2024-05-04T14:26:26.877 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:26.879 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:26.887 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:26.888 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/520) 2024-05-04T14:26:27.035 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:27.041 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:27.041 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:27.155 INFO:teuthology.orchestra.run.smithi052.stdout:generic/520 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T14:26:27.159 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/520 2024-05-04T14:26:27.159 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/520 2024-05-04T14:26:27.160 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T14:26:27.162 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:26:27.170 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T14:26:27.170 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/521) 2024-05-04T14:26:27.318 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T14:26:27.322 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T14:26:27.322 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T14:28:25.287 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:28:25.290 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:28:55.487 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:28:55.491 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:29:25.534 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:29:25.539 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:29:55.582 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:29:55.588 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:30:25.670 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:30:25.674 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:30:55.778 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:30:55.782 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:31:25.818 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:31:25.821 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:31:55.938 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:31:55.942 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:32:25.993 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:32:25.996 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:32:56.040 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:32:56.043 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:33:26.147 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:33:26.151 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:33:56.210 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:33:56.214 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:34:26.308 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:34:26.311 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:34:56.439 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:34:56.442 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:35:26.514 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:35:26.517 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:35:56.566 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:35:56.569 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:36:26.655 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:36:26.658 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:36:26.686 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T14:36:26.680+0000 7f7b9124a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 378875) UID: 0 2024-05-04T14:36:26.686 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T14:36:26.680+0000 7fd3bf2df640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 378875) UID: 0 2024-05-04T14:36:26.687 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T14:36:26.680+0000 7f3c99d9c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 378875) UID: 0 2024-05-04T14:36:26.687 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T14:36:26.680+0000 7f17989d7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 378875) UID: 0 2024-05-04T14:39:04.938 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:39:04.942 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:39:35.003 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:39:35.006 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:40:05.117 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:40:05.120 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:40:35.474 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:40:35.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:41:05.544 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:41:05.548 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:41:35.678 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:41:35.681 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:42:05.724 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:42:05.728 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:42:35.806 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:42:35.809 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:43:06.176 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:43:06.179 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:43:36.225 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:43:36.228 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:44:06.307 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:44:06.310 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:44:36.645 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:44:36.648 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:45:06.703 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:45:06.706 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:45:36.771 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:45:36.774 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:46:06.819 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:46:06.822 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:46:36.943 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:46:36.946 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:47:06.989 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:47:06.993 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:47:37.096 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:47:37.099 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:48:07.146 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:48:07.150 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:48:37.190 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:48:37.193 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:49:07.297 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:49:07.299 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:49:37.368 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:49:37.372 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:50:07.474 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:50:07.478 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:50:37.540 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:50:37.544 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:51:07.588 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:51:07.591 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:51:37.666 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:51:37.669 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:52:07.798 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:52:07.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:52:37.894 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:52:37.897 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:53:08.001 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:53:08.004 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:53:38.089 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:53:38.093 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:54:08.161 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:54:08.165 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:54:38.234 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:54:38.237 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:55:08.305 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:55:08.311 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:55:38.404 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:55:38.408 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:56:08.493 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:56:08.496 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:56:38.607 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:56:38.611 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:57:08.933 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:57:08.937 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:57:38.982 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:57:38.985 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:58:09.023 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:58:09.026 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:58:39.072 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:58:39.075 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:59:09.137 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:59:09.140 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:59:39.199 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T14:59:39.202 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:00:09.255 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:00:09.258 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:00:39.325 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:00:39.329 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:01:09.424 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:01:09.427 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:01:39.513 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:01:39.516 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:02:09.636 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:02:09.639 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:02:39.758 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:02:39.761 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:03:09.802 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:03:09.806 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:03:39.860 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:03:39.864 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:04:09.983 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:04:09.987 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:04:40.057 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:04:40.060 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:05:10.161 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:05:10.165 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:05:40.283 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:05:40.286 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:06:10.330 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:06:10.334 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:06:40.378 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:06:40.381 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:07:10.433 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:07:10.436 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:07:40.481 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:07:40.485 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:08:10.571 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:08:10.574 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:08:10.593 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T15:08:10.591+0000 7f1412d90640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35214) UID: 0 2024-05-04T15:08:10.594 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T15:08:10.591+0000 7fb74db8d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35214) UID: 0 2024-05-04T15:08:10.594 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T15:08:10.591+0000 7fc53dde6640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35214) UID: 0 2024-05-04T15:08:10.594 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T15:08:10.591+0000 7f81dc32b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35214) UID: 0 2024-05-04T15:10:27.057 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:10:27.061 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:10:57.160 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:10:57.163 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:11:27.399 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:11:27.402 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:11:57.448 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:11:57.451 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:12:27.577 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:12:27.581 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:12:57.625 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:12:57.628 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:13:27.673 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:13:27.676 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:13:57.762 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:13:57.765 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:14:27.884 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:14:27.887 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:14:57.982 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:14:57.985 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:15:28.048 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:15:28.051 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:15:58.100 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:15:58.103 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:16:28.274 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:16:28.278 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:16:58.322 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:16:58.325 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:17:28.418 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:17:28.422 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:17:58.516 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:17:58.519 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:18:28.635 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:18:28.638 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:18:58.684 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:18:58.687 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:19:28.725 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:19:28.728 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:19:58.831 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:19:58.834 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:20:28.903 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:20:28.906 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:20:59.297 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:20:59.300 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:21:29.345 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:21:29.347 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:21:59.495 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:21:59.498 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:21:59.518 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T15:21:59.516+0000 7f7b9124a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 379145) UID: 0 2024-05-04T15:21:59.518 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T15:21:59.516+0000 7f17989d7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 379145) UID: 0 2024-05-04T15:21:59.519 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T15:21:59.516+0000 7f3c99d9c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 379145) UID: 0 2024-05-04T15:21:59.519 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T15:21:59.516+0000 7fd3bf2df640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 379145) UID: 0 2024-05-04T15:24:11.355 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:24:11.359 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:24:41.667 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:24:41.670 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:25:11.752 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:25:11.755 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:25:41.816 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:25:41.819 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:26:11.888 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:26:11.891 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:26:41.977 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:26:42.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:27:12.275 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:27:12.278 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:27:42.435 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:27:42.438 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:28:12.703 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:28:12.706 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:28:42.766 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:28:42.770 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:29:12.821 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:29:12.824 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:29:42.885 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:29:42.888 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:30:12.932 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:30:12.935 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:30:43.006 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:30:43.010 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:31:13.103 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:31:13.106 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:31:43.175 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:31:43.179 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:32:13.222 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:32:13.226 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:32:43.287 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:32:43.290 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:33:13.375 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:33:13.378 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:33:43.490 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:33:43.493 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:34:13.604 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:34:13.607 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:34:43.743 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:34:43.747 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:35:13.806 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:35:13.810 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:35:43.879 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:35:43.882 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:36:13.985 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:36:13.989 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:36:44.132 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:36:44.136 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:37:14.247 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:37:14.250 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:37:44.344 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:37:44.347 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:38:14.391 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:38:14.395 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:38:44.438 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:38:44.442 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:39:14.778 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:39:14.781 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:39:44.841 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:39:44.844 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:40:14.972 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:40:14.975 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:40:45.328 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:40:45.331 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:41:15.395 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:41:15.398 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:41:45.438 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:41:45.442 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:42:15.528 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:42:15.530 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:42:45.643 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:42:45.646 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:43:15.690 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:43:15.693 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:43:45.730 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:43:45.734 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:44:15.779 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:44:15.782 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:44:45.844 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:44:45.847 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:45:15.956 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:45:15.959 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:45:46.021 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:45:46.025 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:46:16.091 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:46:16.094 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:46:46.133 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:46:46.136 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:46:46.165 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T15:46:46.160+0000 7f7b9124a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 379305) UID: 0 2024-05-04T15:46:46.166 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T15:46:46.160+0000 7f3c99d9c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 379305) UID: 0 2024-05-04T15:46:46.166 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T15:46:46.160+0000 7f17989d7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 379305) UID: 0 2024-05-04T15:46:46.166 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T15:46:46.160+0000 7fd3bf2df640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 379305) UID: 0 2024-05-04T15:49:03.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:49:03.508 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:49:33.558 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:49:33.561 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:50:03.671 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:50:03.675 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:50:33.786 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:50:33.789 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:51:03.934 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:51:03.937 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:51:34.179 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:51:34.182 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:52:04.295 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:52:04.298 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:52:34.370 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:52:34.373 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:53:04.482 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:53:04.485 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:53:34.588 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:53:34.592 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:54:04.702 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:54:04.706 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:54:34.816 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:54:34.820 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:55:04.939 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:55:04.942 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:55:34.994 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:55:34.997 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:56:05.151 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:56:05.154 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:56:35.225 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:56:35.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:57:05.270 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:57:05.274 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:57:35.317 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:57:35.320 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:58:05.363 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:58:05.366 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:58:35.411 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:58:35.414 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:59:05.459 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:59:05.462 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:59:35.500 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T15:59:35.503 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:00:05.588 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:00:05.591 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:00:35.642 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:00:35.646 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:01:05.707 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:01:05.710 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:01:35.779 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:01:35.783 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:02:06.244 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:02:06.248 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:02:36.301 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:02:36.305 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:03:06.381 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:03:06.384 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:03:23.287 INFO:teuthology.orchestra.run.smithi052.stdout:generic/521 5816s 2024-05-04T16:03:23.332 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/521 2024-05-04T16:03:23.332 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T16:03:23.337 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T16:03:23.351 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T16:03:23.352 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/522) 2024-05-04T16:03:24.264 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T16:03:24.270 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T16:03:24.270 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T16:03:36.729 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:03:36.732 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:04:06.843 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:04:06.846 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:04:37.173 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:04:37.176 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:05:07.263 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:05:07.266 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:05:37.318 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:05:37.321 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:06:07.425 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:06:07.428 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:06:37.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:06:37.508 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:07:07.610 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:07:07.614 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:07:37.751 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:07:37.754 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:08:07.797 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:08:07.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:08:37.852 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:08:37.856 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:09:07.933 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:09:07.936 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:09:38.073 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:09:38.076 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:10:08.170 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:10:08.173 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:10:38.284 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:10:38.288 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:11:08.348 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:11:08.351 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:11:38.437 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:11:38.441 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:12:08.552 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:12:08.555 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:12:38.650 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:12:38.653 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:13:08.755 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:13:08.758 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:13:38.886 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:13:38.889 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:14:08.941 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:14:08.944 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:14:38.997 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:14:39.000 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:15:09.111 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:15:09.114 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:15:39.168 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:15:39.171 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:16:09.232 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:16:09.235 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:16:39.347 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:16:39.349 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:17:09.415 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:17:09.418 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:17:39.467 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:17:39.470 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:18:09.515 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:18:09.518 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:18:39.595 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:18:39.598 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:19:09.728 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:19:09.731 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:19:39.866 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:19:39.869 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:20:09.912 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:20:09.916 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:20:39.958 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:20:39.961 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:21:10.034 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:21:10.036 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:21:40.156 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:21:40.159 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:22:10.269 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:22:10.273 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:22:40.320 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:22:40.324 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:23:10.384 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:23:10.387 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:23:40.433 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:23:40.436 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:23:40.454 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T16:23:40.452+0000 7fbf91e1b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 379974) UID: 0 2024-05-04T16:23:40.455 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-04T16:23:40.452+0000 7f6f26d7f640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 379974) UID: 0 2024-05-04T16:26:15.442 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:26:15.446 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:26:45.507 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:26:45.510 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:27:15.550 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:27:15.553 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:27:45.681 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:27:45.684 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:28:15.737 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:28:15.740 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:28:45.817 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:28:45.821 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:29:15.940 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:29:15.943 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:29:46.296 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:29:46.299 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:30:16.456 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:30:16.459 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:30:46.506 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:30:46.510 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:31:16.613 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:31:16.617 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:31:46.687 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:31:46.690 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:32:16.792 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:32:16.795 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:32:46.905 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:32:46.909 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:33:16.961 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:33:16.964 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:33:47.101 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:33:47.104 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:34:17.146 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:34:17.149 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:34:47.271 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:34:47.274 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:35:17.384 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:35:17.388 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:35:47.440 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:35:47.443 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:36:17.488 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:36:17.491 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:36:47.559 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:36:47.563 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:37:17.650 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:37:17.653 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:37:47.713 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:37:47.717 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:38:17.802 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:38:17.805 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:38:47.992 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:38:47.995 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:39:18.038 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:39:18.041 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:39:48.170 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:39:48.174 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:40:18.218 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:40:18.221 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:40:48.315 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:40:48.319 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:41:18.370 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:41:18.373 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:41:48.452 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:41:48.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:42:18.550 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:42:18.555 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:42:48.710 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:42:48.713 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:43:18.756 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:43:18.760 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:43:48.883 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:43:48.886 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:44:18.980 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:44:18.984 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:44:49.069 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:44:49.073 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:45:19.117 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:45:19.120 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:45:49.173 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:45:49.176 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:46:19.220 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:46:19.223 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:46:49.452 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:46:49.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:47:19.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:47:19.509 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:47:49.588 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:47:49.591 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:48:19.635 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:48:19.639 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:48:49.693 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:48:49.697 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:49:19.955 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:49:19.958 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:49:50.005 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:49:50.008 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:50:20.101 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:50:20.104 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:50:50.216 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:50:50.219 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:51:20.262 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:51:20.265 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:51:50.336 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:51:50.341 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:52:20.376 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:52:20.380 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:52:50.472 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:52:50.476 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:53:20.614 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:53:20.617 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:53:50.692 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:53:50.695 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:54:20.764 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:54:20.768 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:54:50.837 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:54:50.840 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:55:20.950 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:55:20.954 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:55:51.065 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:55:51.068 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:56:21.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:56:21.116 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:56:51.185 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:56:51.188 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:57:21.232 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:57:21.236 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:57:51.294 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:57:51.298 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:58:21.344 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:58:21.347 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:58:51.450 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:58:51.453 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:59:21.582 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:59:21.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:59:51.696 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T16:59:51.699 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:00:21.745 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:00:21.749 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:00:51.881 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:00:51.884 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:01:21.979 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:01:21.982 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:01:52.051 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:01:52.054 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:02:22.141 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:02:22.144 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:02:52.188 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:02:52.191 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:03:22.732 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:03:22.735 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:03:52.775 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:03:52.781 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:04:22.830 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:04:22.832 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:04:52.928 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:04:52.931 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:05:23.050 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:05:23.053 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:05:53.099 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:05:53.103 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:06:23.170 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:06:23.173 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:06:53.235 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:06:53.238 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:07:23.281 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:07:23.284 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:07:53.337 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:07:53.345 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:08:23.428 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:08:23.430 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:08:53.549 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:08:53.552 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:09:23.630 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:09:23.634 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:09:53.678 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:09:53.681 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:09:53.711 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T17:09:53.705+0000 7fb74db8d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35935) UID: 0 2024-05-04T17:09:53.711 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T17:09:53.705+0000 7f1412d90640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35935) UID: 0 2024-05-04T17:09:53.711 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T17:09:53.705+0000 7fc53dde6640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35935) UID: 0 2024-05-04T17:09:53.712 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T17:09:53.705+0000 7f81dc32b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35935) UID: 0 2024-05-04T17:12:09.943 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:12:09.946 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:12:39.998 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:12:40.001 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:13:10.039 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:13:10.042 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:13:40.086 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:13:40.089 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:14:10.226 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:14:10.230 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:14:40.306 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:14:40.355 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:15:10.396 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:15:10.400 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:15:40.451 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:15:40.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:15:40.473 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T17:15:40.467+0000 7f81dc32b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35967) UID: 0 2024-05-04T17:15:40.473 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T17:15:40.467+0000 7f1412d90640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35967) UID: 0 2024-05-04T17:15:40.473 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T17:15:40.467+0000 7fb74db8d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35967) UID: 0 2024-05-04T17:15:40.474 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T17:15:40.467+0000 7fc53dde6640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35967) UID: 0 2024-05-04T17:18:07.138 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:18:07.142 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:18:37.276 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:18:37.279 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:19:07.490 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:19:07.493 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:19:37.546 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:19:37.549 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:20:07.660 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:20:07.663 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:20:37.705 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:20:37.708 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:20:50.774 INFO:teuthology.orchestra.run.smithi052.stdout:generic/522 4646s 2024-05-04T17:20:50.788 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/522 2024-05-04T17:20:50.789 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:50.793 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:50.852 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:50.852 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/523) 2024-05-04T17:20:51.171 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:51.178 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:51.178 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:51.546 INFO:teuthology.orchestra.run.smithi052.stdout:generic/523 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T17:20:51.551 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/523 2024-05-04T17:20:51.551 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/523 2024-05-04T17:20:51.551 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:51.554 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:51.566 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:51.567 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/524) 2024-05-04T17:20:51.763 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:51.768 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:51.768 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:generic/524 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T17:20:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/524 2024-05-04T17:20:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/524 2024-05-04T17:20:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:51.925 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:51.935 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:51.935 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/525) 2024-05-04T17:20:52.118 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:52.122 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:52.122 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:52.247 INFO:teuthology.orchestra.run.smithi052.stdout:generic/525 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T17:20:52.251 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/525 2024-05-04T17:20:52.251 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/525 2024-05-04T17:20:52.251 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:52.254 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:52.264 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:52.264 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/526) 2024-05-04T17:20:52.443 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:52.448 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:52.448 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:52.637 INFO:teuthology.orchestra.run.smithi052.stdout:generic/526 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T17:20:52.642 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/526 2024-05-04T17:20:52.643 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/526 2024-05-04T17:20:52.643 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:52.646 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:52.658 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:52.658 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/527) 2024-05-04T17:20:52.850 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:52.854 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:52.855 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:52.993 INFO:teuthology.orchestra.run.smithi052.stdout:generic/527 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T17:20:52.996 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/527 2024-05-04T17:20:52.996 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/527 2024-05-04T17:20:52.996 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:52.999 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:53.008 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:53.008 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/528) 2024-05-04T17:20:53.168 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:53.172 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:53.173 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:53.312 INFO:teuthology.orchestra.run.smithi052.stdout:generic/528 [not run] inode creation time not supported by this filesystem 2024-05-04T17:20:53.316 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/528 2024-05-04T17:20:53.316 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/528 2024-05-04T17:20:53.316 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:53.319 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:53.328 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:53.328 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/529) 2024-05-04T17:20:53.504 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:53.509 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:53.509 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:53.666 INFO:teuthology.orchestra.run.smithi052.stdout:generic/529 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-05-04T17:20:53.669 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/529 2024-05-04T17:20:53.669 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/529 2024-05-04T17:20:53.669 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:53.672 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:53.681 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:53.682 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/530) 2024-05-04T17:20:53.874 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:53.878 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:53.878 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:53.985 INFO:teuthology.orchestra.run.smithi052.stdout:generic/530 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T17:20:53.988 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/530 2024-05-04T17:20:53.989 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/530 2024-05-04T17:20:53.989 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:53.991 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:53.999 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:53.999 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/531) 2024-05-04T17:20:54.149 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:54.153 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:54.153 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:54.254 INFO:teuthology.orchestra.run.smithi052.stdout:generic/531 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T17:20:54.258 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/531 2024-05-04T17:20:54.258 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/531 2024-05-04T17:20:54.258 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:54.260 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:54.268 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:54.268 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/532) 2024-05-04T17:20:54.417 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:54.421 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:54.421 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:54.588 INFO:teuthology.orchestra.run.smithi052.stdout:generic/532 0s 2024-05-04T17:20:54.593 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/532 2024-05-04T17:20:54.593 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:54.596 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:54.604 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:54.604 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/533) 2024-05-04T17:20:54.756 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:54.760 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:54.760 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:55.134 INFO:teuthology.orchestra.run.smithi052.stdout:generic/533 1s 2024-05-04T17:20:55.139 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/533 2024-05-04T17:20:55.139 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:55.141 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:55.149 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:55.149 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/534) 2024-05-04T17:20:55.303 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:55.307 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:55.307 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:55.431 INFO:teuthology.orchestra.run.smithi052.stdout:generic/534 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T17:20:55.435 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/534 2024-05-04T17:20:55.435 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/534 2024-05-04T17:20:55.435 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:55.437 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:55.445 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:55.446 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/535) 2024-05-04T17:20:55.599 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:55.603 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:55.603 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:55.731 INFO:teuthology.orchestra.run.smithi052.stdout:generic/535 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T17:20:55.735 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/535 2024-05-04T17:20:55.735 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/535 2024-05-04T17:20:55.735 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:55.737 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:55.745 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:55.746 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/536) 2024-05-04T17:20:55.900 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:55.904 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:55.904 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:56.003 INFO:teuthology.orchestra.run.smithi052.stdout:generic/536 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T17:20:56.007 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/536 2024-05-04T17:20:56.007 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/536 2024-05-04T17:20:56.007 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:56.009 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:56.018 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:56.018 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/537) 2024-05-04T17:20:56.169 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:56.173 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:56.173 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:56.277 INFO:teuthology.orchestra.run.smithi052.stdout:generic/537 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T17:20:56.281 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/537 2024-05-04T17:20:56.281 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/537 2024-05-04T17:20:56.281 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T17:20:56.283 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:20:56.291 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T17:20:56.291 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/538) 2024-05-04T17:20:56.443 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T17:20:56.447 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T17:20:56.447 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T17:21:07.820 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:21:07.823 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:21:37.942 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:21:37.946 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:22:07.990 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:22:07.993 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:22:38.053 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:22:38.056 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:23:08.134 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:23:08.138 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:23:38.207 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:23:38.211 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:24:08.312 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:24:08.316 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:24:38.443 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:24:38.446 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:25:08.532 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:25:08.535 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:25:38.588 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:25:38.592 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:26:08.635 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:26:08.639 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:26:38.699 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:26:38.702 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:27:08.805 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:27:08.808 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:27:38.878 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:27:38.881 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:28:09.125 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:28:09.129 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:28:39.200 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:28:39.203 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:29:09.277 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:29:09.281 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:29:39.400 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:29:39.403 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:30:09.452 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:30:09.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:30:39.562 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:30:39.565 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:31:09.642 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:31:09.645 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:31:39.697 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:31:39.701 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:32:10.029 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:32:10.032 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:32:40.292 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:32:40.296 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:33:10.348 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:33:10.352 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:33:40.462 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:33:40.466 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:34:10.592 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:34:10.595 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:34:40.782 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:34:40.788 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:35:10.828 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:35:10.831 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:35:40.883 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:35:40.886 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:36:10.982 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:36:10.985 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:36:41.088 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:36:41.091 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:37:11.194 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:37:11.197 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:37:11.215 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T17:37:11.212+0000 7f7b9124a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 388387) UID: 0 2024-05-04T17:37:11.215 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T17:37:11.212+0000 7f3c99d9c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 388387) UID: 0 2024-05-04T17:37:11.215 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T17:37:11.212+0000 7f17989d7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 388387) UID: 0 2024-05-04T17:37:11.216 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T17:37:11.212+0000 7fd3bf2df640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 388387) UID: 0 2024-05-04T17:39:43.681 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:39:43.684 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:40:13.741 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:40:13.745 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:40:43.807 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:40:43.810 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:41:13.919 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:41:13.922 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:41:43.992 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:41:43.996 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:42:14.106 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:42:14.109 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:42:44.186 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:42:44.191 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:43:14.245 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:43:14.249 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:43:44.323 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:43:44.327 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:44:14.463 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:44:14.466 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:44:44.576 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:44:44.579 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:45:14.617 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:45:14.621 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:45:44.670 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:45:44.673 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:46:14.717 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:46:14.720 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:46:44.840 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:46:44.844 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:47:14.963 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:47:14.966 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:47:45.027 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:47:45.031 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:48:15.232 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:48:15.235 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:48:45.280 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:48:45.283 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:49:15.394 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:49:15.397 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:49:45.508 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:49:45.511 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:50:15.622 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:50:15.625 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:50:45.727 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:50:45.730 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:51:15.783 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:51:15.786 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:51:46.104 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:51:46.108 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:52:16.186 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:52:16.189 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:52:46.266 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:52:46.269 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:53:16.313 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:53:16.316 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:53:46.436 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:53:46.439 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:54:16.483 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:54:16.486 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:54:46.539 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:54:46.542 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:55:16.611 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:55:16.614 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:55:46.742 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:55:46.745 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:56:16.842 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:56:16.845 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:56:46.973 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:56:46.976 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:57:17.025 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:57:17.029 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:57:47.091 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:57:47.094 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:58:17.165 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:58:17.168 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:58:47.354 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:58:47.357 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:59:17.425 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:59:17.429 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:59:47.598 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T17:59:47.601 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:00:17.646 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:00:17.649 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:00:47.801 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:00:47.804 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:01:17.848 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:01:17.851 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:01:47.987 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:01:47.990 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:02:18.259 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:02:18.263 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:02:48.341 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:02:48.344 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:03:18.387 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:03:18.391 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:03:48.501 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:03:48.504 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:04:18.557 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:04:18.560 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:04:48.640 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:04:48.643 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:05:18.690 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:05:18.693 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:05:48.808 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:05:48.811 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:06:18.946 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:06:18.950 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:06:49.060 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:06:49.064 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:07:19.108 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:07:19.111 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:07:49.171 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:07:49.174 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:08:19.285 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:08:19.288 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:08:49.351 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:08:49.354 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:09:19.431 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:09:19.434 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:09:49.570 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:09:49.573 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:10:19.917 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:10:19.920 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:10:49.970 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:10:49.974 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:11:20.120 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:11:20.123 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:11:50.260 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:11:50.263 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:12:20.382 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:12:20.384 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:12:50.745 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:12:50.748 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:13:20.800 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:13:20.803 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:13:50.856 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:13:50.859 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:14:20.920 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:14:20.923 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:14:51.051 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:14:51.054 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:15:21.198 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:15:21.201 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:15:51.255 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:15:51.258 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:16:21.309 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:16:21.312 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:16:51.624 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:16:51.627 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:17:21.679 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:17:21.683 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:17:51.794 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:17:51.797 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:18:21.942 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:18:21.945 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:18:51.997 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:18:52.003 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:19:22.095 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:19:22.099 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:19:52.377 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:19:52.380 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:20:22.422 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:20:22.425 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:20:52.578 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:20:52.581 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:21:22.620 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:21:22.624 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:21:52.747 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:21:52.750 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:22:22.879 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:22:22.882 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:22:53.234 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:22:53.239 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:23:23.281 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:23:23.288 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:23:53.345 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:23:53.348 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:24:23.451 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:24:23.454 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:24:53.581 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:24:53.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:25:23.712 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:25:23.716 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:25:53.835 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:25:53.839 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:26:23.882 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:26:23.885 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:26:53.955 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:26:53.958 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:27:24.069 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:27:24.072 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:27:54.124 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:27:54.128 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:28:24.171 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:28:24.174 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:28:54.461 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:28:54.464 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:29:24.508 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:29:24.511 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:29:54.577 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:29:54.580 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:30:24.678 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:30:24.681 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:30:54.817 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:30:54.820 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:31:24.865 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:31:24.868 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:31:54.962 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:31:54.965 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:32:25.042 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:32:25.045 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:32:55.123 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:32:55.126 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:33:25.246 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:33:25.249 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:33:55.305 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:33:55.308 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:34:25.423 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:34:25.427 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:34:55.530 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:34:55.533 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:35:25.601 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:35:25.604 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:35:55.658 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:35:55.661 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:36:25.704 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:36:25.707 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:36:55.818 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:36:55.821 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:37:25.949 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:37:25.952 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:37:56.005 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:37:56.017 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:38:26.079 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:38:26.082 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:38:56.133 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:38:56.136 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:39:26.213 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:39:26.216 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:39:56.319 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:39:56.322 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:39:56.342 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T18:39:56.340+0000 7f7b9124a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 393092) UID: 0 2024-05-04T18:39:56.342 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T18:39:56.340+0000 7f3c99d9c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 393092) UID: 0 2024-05-04T18:39:56.342 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T18:39:56.340+0000 7fd3bf2df640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 393092) UID: 0 2024-05-04T18:39:56.343 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T18:39:56.340+0000 7f17989d7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 393092) UID: 0 2024-05-04T18:42:21.322 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:42:21.326 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:42:51.376 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:42:51.379 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:43:21.440 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:43:21.443 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:43:51.495 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:43:51.499 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:44:21.544 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:44:21.547 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:44:51.598 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:44:51.601 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:45:21.704 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:45:21.707 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:45:51.752 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:45:51.755 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:46:21.793 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:46:21.797 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:46:51.872 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:46:51.876 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:47:21.994 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:47:21.997 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:47:52.125 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:47:52.128 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:48:22.197 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:48:22.201 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:48:52.437 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:48:52.439 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:49:22.484 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:49:22.487 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:49:52.539 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:49:52.542 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:50:16.363 INFO:teuthology.orchestra.run.smithi052.stdout:generic/538 5360s 2024-05-04T18:50:16.378 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/538 2024-05-04T18:50:16.378 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:16.381 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:16.392 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:16.393 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/539) 2024-05-04T18:50:16.951 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:16.955 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:16.955 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:20.245 INFO:teuthology.orchestra.run.smithi052.stdout:generic/539 4s 2024-05-04T18:50:20.252 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/539 2024-05-04T18:50:20.253 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:20.256 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:20.268 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:20.269 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/540) 2024-05-04T18:50:20.836 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:20.841 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:20.841 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:20.963 INFO:teuthology.orchestra.run.smithi052.stdout:generic/540 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:20.967 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/540 2024-05-04T18:50:20.968 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/540 2024-05-04T18:50:20.968 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:20.970 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:20.980 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:20.980 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/541) 2024-05-04T18:50:21.165 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:21.170 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:21.170 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:21.293 INFO:teuthology.orchestra.run.smithi052.stdout:generic/541 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:21.297 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/541 2024-05-04T18:50:21.297 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/541 2024-05-04T18:50:21.297 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:21.300 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:21.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:21.310 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/542) 2024-05-04T18:50:21.490 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:21.495 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:21.495 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:21.614 INFO:teuthology.orchestra.run.smithi052.stdout:generic/542 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:21.618 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/542 2024-05-04T18:50:21.619 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/542 2024-05-04T18:50:21.619 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:21.621 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:21.630 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:21.630 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/543) 2024-05-04T18:50:21.795 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:21.799 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:21.799 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:generic/543 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:21.907 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/543 2024-05-04T18:50:21.907 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/543 2024-05-04T18:50:21.907 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:21.910 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:21.917 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:21.918 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/544) 2024-05-04T18:50:22.069 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:22.073 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:22.073 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:22.179 INFO:teuthology.orchestra.run.smithi052.stdout:generic/544 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:22.182 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/544 2024-05-04T18:50:22.182 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/544 2024-05-04T18:50:22.182 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:22.184 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:22.192 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:22.192 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/545) 2024-05-04T18:50:22.343 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:22.348 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:22.348 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:22.653 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:50:22.655 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:50:22.877 INFO:teuthology.orchestra.run.smithi052.stdout:generic/545 [not run] file system doesn't support chattr +i 2024-05-04T18:50:22.884 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/545 2024-05-04T18:50:22.884 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/545 2024-05-04T18:50:22.884 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:22.889 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:22.901 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:22.902 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/546) 2024-05-04T18:50:23.104 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:23.109 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:23.109 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:23.320 INFO:teuthology.orchestra.run.smithi052.stdout:generic/546 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:23.325 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/546 2024-05-04T18:50:23.325 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/546 2024-05-04T18:50:23.325 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:23.328 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:23.339 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:23.339 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/547) 2024-05-04T18:50:23.533 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:23.538 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:23.538 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:23.705 INFO:teuthology.orchestra.run.smithi052.stdout:generic/547 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:23.710 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/547 2024-05-04T18:50:23.710 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/547 2024-05-04T18:50:23.710 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:23.713 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:23.724 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:23.724 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/548) 2024-05-04T18:50:23.908 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:23.912 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:23.913 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:24.068 INFO:teuthology.orchestra.run.smithi052.stdout:generic/548 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:24.072 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/548 2024-05-04T18:50:24.072 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/548 2024-05-04T18:50:24.073 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:24.075 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:24.083 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:24.084 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/549) 2024-05-04T18:50:24.253 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:24.258 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:24.258 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:24.391 INFO:teuthology.orchestra.run.smithi052.stdout:generic/549 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:24.395 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/549 2024-05-04T18:50:24.395 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/549 2024-05-04T18:50:24.395 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:24.397 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:24.405 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:24.406 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/550) 2024-05-04T18:50:24.560 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:24.565 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:24.565 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:24.713 INFO:teuthology.orchestra.run.smithi052.stdout:generic/550 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:24.716 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/550 2024-05-04T18:50:24.716 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/550 2024-05-04T18:50:24.717 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:24.719 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:24.727 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:24.727 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/551) 2024-05-04T18:50:24.882 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:24.887 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:24.887 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:24.990 INFO:teuthology.orchestra.run.smithi052.stdout:generic/551 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:24.993 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/551 2024-05-04T18:50:24.993 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/551 2024-05-04T18:50:24.993 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:24.996 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:25.003 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:25.003 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/552) 2024-05-04T18:50:25.156 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:25.160 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:25.160 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:25.279 INFO:teuthology.orchestra.run.smithi052.stdout:generic/552 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:25.283 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/552 2024-05-04T18:50:25.283 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/552 2024-05-04T18:50:25.283 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:25.285 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:25.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:25.293 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/553) 2024-05-04T18:50:25.447 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:25.451 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:25.451 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:25.625 INFO:teuthology.orchestra.run.smithi052.stdout:generic/553 [not run] xfs_io chattr +i failed (old kernel/wrong fs/bad args?) 2024-05-04T18:50:25.629 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/553 2024-05-04T18:50:25.629 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/553 2024-05-04T18:50:25.629 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:25.631 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:25.640 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:25.640 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/554) 2024-05-04T18:50:26.133 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:26.137 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:26.137 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:26.240 INFO:teuthology.orchestra.run.smithi052.stdout:generic/554 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:26.243 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/554 2024-05-04T18:50:26.244 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/554 2024-05-04T18:50:26.244 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:26.246 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:26.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:26.253 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/555) 2024-05-04T18:50:26.401 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:26.406 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:26.406 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:26.546 INFO:teuthology.orchestra.run.smithi052.stdout:generic/555 [not run] xfs_io chattr +ia failed (old kernel/wrong fs/bad args?) 2024-05-04T18:50:26.550 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/555 2024-05-04T18:50:26.550 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/555 2024-05-04T18:50:26.550 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:26.552 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:26.560 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:26.560 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/556) 2024-05-04T18:50:27.053 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:27.057 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:27.057 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:27.186 INFO:teuthology.orchestra.run.smithi052.stdout:generic/556 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:27.189 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/556 2024-05-04T18:50:27.189 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/556 2024-05-04T18:50:27.189 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:27.192 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:27.199 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:27.199 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/557) 2024-05-04T18:50:27.350 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:27.355 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:27.355 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:27.499 INFO:teuthology.orchestra.run.smithi052.stdout:generic/557 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:27.503 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/557 2024-05-04T18:50:27.503 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/557 2024-05-04T18:50:27.503 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:27.506 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:27.515 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:27.515 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/558) 2024-05-04T18:50:27.669 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:27.673 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:27.673 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:27.798 INFO:teuthology.orchestra.run.smithi052.stdout:generic/558 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:27.801 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/558 2024-05-04T18:50:27.801 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/558 2024-05-04T18:50:27.801 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:27.804 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:27.812 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:27.812 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/559) 2024-05-04T18:50:27.967 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:27.972 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:27.972 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:28.085 INFO:teuthology.orchestra.run.smithi052.stdout:generic/559 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:28.088 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/559 2024-05-04T18:50:28.089 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/559 2024-05-04T18:50:28.089 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:28.091 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:28.099 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:28.099 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/560) 2024-05-04T18:50:28.254 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:28.261 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:28.262 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:28.383 INFO:teuthology.orchestra.run.smithi052.stdout:generic/560 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:28.386 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/560 2024-05-04T18:50:28.386 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/560 2024-05-04T18:50:28.387 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:28.389 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:28.397 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:28.397 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/561) 2024-05-04T18:50:28.546 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:28.551 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:28.551 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:28.655 INFO:teuthology.orchestra.run.smithi052.stdout:generic/561 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:28.658 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/561 2024-05-04T18:50:28.659 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/561 2024-05-04T18:50:28.659 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:28.661 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:28.670 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:28.670 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/562) 2024-05-04T18:50:28.819 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:28.823 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:28.823 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:28.927 INFO:teuthology.orchestra.run.smithi052.stdout:generic/562 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:28.932 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/562 2024-05-04T18:50:28.932 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/562 2024-05-04T18:50:28.932 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:28.935 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:28.943 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:28.943 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/563) 2024-05-04T18:50:29.095 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:29.099 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:29.099 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:29.246 INFO:teuthology.orchestra.run.smithi052.stdout:generic/563 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:29.250 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/563 2024-05-04T18:50:29.251 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/563 2024-05-04T18:50:29.251 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:29.252 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:29.260 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:29.260 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/564) 2024-05-04T18:50:29.409 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:29.413 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:29.414 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:30.091 INFO:teuthology.orchestra.run.smithi052.stdout:generic/564 1s 2024-05-04T18:50:30.098 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/564 2024-05-04T18:50:30.098 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:30.099 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:30.231 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:30.231 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/565) 2024-05-04T18:50:30.631 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:30.635 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:30.635 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:30.740 INFO:teuthology.orchestra.run.smithi052.stdout:generic/565 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:30.744 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/565 2024-05-04T18:50:30.744 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/565 2024-05-04T18:50:30.744 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:30.747 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:30.754 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:30.755 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/566) 2024-05-04T18:50:30.903 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:30.908 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:30.908 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:31.015 INFO:teuthology.orchestra.run.smithi052.stdout:generic/566 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:31.018 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/566 2024-05-04T18:50:31.019 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/566 2024-05-04T18:50:31.019 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:31.021 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:31.029 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:31.029 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/567) 2024-05-04T18:50:31.185 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:31.189 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:31.189 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:31.292 INFO:teuthology.orchestra.run.smithi052.stdout:generic/567 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:31.296 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/567 2024-05-04T18:50:31.296 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/567 2024-05-04T18:50:31.297 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:31.298 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:31.306 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:31.307 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/568) 2024-05-04T18:50:31.460 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:31.464 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:31.464 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:31.649 INFO:teuthology.orchestra.run.smithi052.stdout:generic/568 0s 2024-05-04T18:50:31.654 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/568 2024-05-04T18:50:31.654 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:31.657 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:31.814 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:31.814 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/569) 2024-05-04T18:50:32.172 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:32.176 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:32.176 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:32.319 INFO:teuthology.orchestra.run.smithi052.stdout:generic/569 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:32.323 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/569 2024-05-04T18:50:32.323 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/569 2024-05-04T18:50:32.323 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:32.325 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:32.333 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:32.333 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/570) 2024-05-04T18:50:32.486 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:32.489 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:32.489 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:32.594 INFO:teuthology.orchestra.run.smithi052.stdout:generic/570 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:32.598 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/570 2024-05-04T18:50:32.598 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/570 2024-05-04T18:50:32.598 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:32.600 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:32.608 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:32.608 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/571) 2024-05-04T18:50:32.758 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:32.762 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:32.762 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:generic/571 6s 2024-05-04T18:50:39.009 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/571 2024-05-04T18:50:39.009 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:39.013 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:39.223 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:39.223 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/572) 2024-05-04T18:50:39.553 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:39.557 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:39.557 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:39.664 INFO:teuthology.orchestra.run.smithi052.stdout:generic/572 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:39.667 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/572 2024-05-04T18:50:39.668 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/572 2024-05-04T18:50:39.668 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:39.678 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:39.678 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/573) 2024-05-04T18:50:39.828 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:39.832 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:39.832 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:39.935 INFO:teuthology.orchestra.run.smithi052.stdout:generic/573 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:39.938 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/573 2024-05-04T18:50:39.938 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/573 2024-05-04T18:50:39.938 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:39.940 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:39.948 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:39.948 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/574) 2024-05-04T18:50:40.098 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:40.102 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:40.103 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:40.205 INFO:teuthology.orchestra.run.smithi052.stdout:generic/574 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:40.208 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/574 2024-05-04T18:50:40.208 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/574 2024-05-04T18:50:40.208 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:40.211 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:40.218 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:40.218 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/575) 2024-05-04T18:50:40.368 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:40.372 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:40.372 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:40.474 INFO:teuthology.orchestra.run.smithi052.stdout:generic/575 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:40.478 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/575 2024-05-04T18:50:40.478 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/575 2024-05-04T18:50:40.478 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:40.480 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:40.488 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:40.488 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/576) 2024-05-04T18:50:40.639 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:40.643 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:40.644 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:40.772 INFO:teuthology.orchestra.run.smithi052.stdout:generic/576 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:40.776 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/576 2024-05-04T18:50:40.776 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/576 2024-05-04T18:50:40.776 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:40.778 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:40.786 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:40.786 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/577) 2024-05-04T18:50:40.936 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:40.940 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:40.940 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:41.043 INFO:teuthology.orchestra.run.smithi052.stdout:generic/577 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:41.046 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/577 2024-05-04T18:50:41.047 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/577 2024-05-04T18:50:41.047 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:41.049 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:41.057 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:41.057 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/578) 2024-05-04T18:50:41.206 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:41.210 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:41.210 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:41.353 INFO:teuthology.orchestra.run.smithi052.stdout:generic/578 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-04T18:50:41.357 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/578 2024-05-04T18:50:41.357 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/578 2024-05-04T18:50:41.357 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:41.360 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:41.367 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:41.367 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/579) 2024-05-04T18:50:41.517 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:41.521 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:41.521 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:41.634 INFO:teuthology.orchestra.run.smithi052.stdout:generic/579 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:41.638 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/579 2024-05-04T18:50:41.638 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/579 2024-05-04T18:50:41.638 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:41.640 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:41.649 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:41.649 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/580) 2024-05-04T18:50:41.809 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:41.813 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:41.813 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:41.940 INFO:teuthology.orchestra.run.smithi052.stdout:generic/580 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:41.944 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/580 2024-05-04T18:50:41.944 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/580 2024-05-04T18:50:41.944 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:41.946 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:41.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:41.955 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/581) 2024-05-04T18:50:42.108 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:42.112 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:42.112 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:42.641 INFO:teuthology.orchestra.run.smithi052.stdout:generic/581 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:42.646 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/581 2024-05-04T18:50:42.646 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/581 2024-05-04T18:50:42.646 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:42.650 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:42.660 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:42.661 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/582) 2024-05-04T18:50:42.846 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:42.851 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:42.851 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:42.993 INFO:teuthology.orchestra.run.smithi052.stdout:generic/582 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:42.997 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/582 2024-05-04T18:50:42.997 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/582 2024-05-04T18:50:42.998 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:43.000 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:43.008 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:43.008 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/583) 2024-05-04T18:50:43.169 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:43.174 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:43.174 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:43.309 INFO:teuthology.orchestra.run.smithi052.stdout:generic/583 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:43.314 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/583 2024-05-04T18:50:43.314 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/583 2024-05-04T18:50:43.314 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:43.316 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:43.325 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:43.325 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/584) 2024-05-04T18:50:43.481 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:43.485 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:43.485 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:43.614 INFO:teuthology.orchestra.run.smithi052.stdout:generic/584 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:43.618 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/584 2024-05-04T18:50:43.618 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/584 2024-05-04T18:50:43.618 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:43.620 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:43.628 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:43.629 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/585) 2024-05-04T18:50:43.780 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:43.786 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:43.787 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:43.893 INFO:teuthology.orchestra.run.smithi052.stdout:generic/585 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:50:43.896 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/585 2024-05-04T18:50:43.897 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/585 2024-05-04T18:50:43.897 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:50:43.899 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:43.906 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:50:43.907 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/586) 2024-05-04T18:50:44.059 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:50:44.063 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:50:44.063 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:50:52.944 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:50:52.946 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:51:22.991 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:51:22.994 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:51:27.715 INFO:teuthology.orchestra.run.smithi052.stdout:generic/586 43s 2024-05-04T18:51:27.724 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/586 2024-05-04T18:51:27.724 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:27.728 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:27.739 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:27.740 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/587) 2024-05-04T18:51:28.288 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:28.292 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:28.292 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:28.490 INFO:teuthology.orchestra.run.smithi052.stdout:generic/587 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-05-04T18:51:28.494 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/587 2024-05-04T18:51:28.494 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/587 2024-05-04T18:51:28.494 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:28.496 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:28.504 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:28.505 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/588) 2024-05-04T18:51:28.663 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:28.667 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:28.667 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:28.784 INFO:teuthology.orchestra.run.smithi052.stdout:generic/588 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:28.787 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/588 2024-05-04T18:51:28.787 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/588 2024-05-04T18:51:28.787 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:28.790 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:28.798 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:28.798 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/589) 2024-05-04T18:51:28.947 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:28.951 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:28.952 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:29.060 INFO:teuthology.orchestra.run.smithi052.stdout:generic/589 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:29.064 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/589 2024-05-04T18:51:29.064 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/589 2024-05-04T18:51:29.064 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:29.066 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:29.074 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:29.074 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/590) 2024-05-04T18:51:29.226 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:29.230 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:29.230 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:29.335 INFO:teuthology.orchestra.run.smithi052.stdout:generic/590 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:29.339 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/590 2024-05-04T18:51:29.339 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/590 2024-05-04T18:51:29.339 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:29.341 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:29.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:29.349 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/591) 2024-05-04T18:51:29.501 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:29.506 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:29.506 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:29.879 INFO:teuthology.orchestra.run.smithi052.stdout:generic/591 0s 2024-05-04T18:51:29.885 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/591 2024-05-04T18:51:29.885 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:29.888 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:29.897 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:29.897 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/592) 2024-05-04T18:51:30.418 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:30.422 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:30.422 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:30.554 INFO:teuthology.orchestra.run.smithi052.stdout:generic/592 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:30.558 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/592 2024-05-04T18:51:30.558 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/592 2024-05-04T18:51:30.558 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:30.560 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:30.568 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:30.568 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/593) 2024-05-04T18:51:30.720 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:30.724 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:30.724 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:30.853 INFO:teuthology.orchestra.run.smithi052.stdout:generic/593 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:30.857 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/593 2024-05-04T18:51:30.857 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/593 2024-05-04T18:51:30.857 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:30.859 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:30.867 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:30.867 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/594) 2024-05-04T18:51:31.019 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:31.023 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:31.023 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:31.127 INFO:teuthology.orchestra.run.smithi052.stdout:generic/594 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:31.131 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/594 2024-05-04T18:51:31.132 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/594 2024-05-04T18:51:31.132 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:31.134 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:31.142 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:31.142 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/595) 2024-05-04T18:51:31.294 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:31.298 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:31.298 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:31.434 INFO:teuthology.orchestra.run.smithi052.stdout:generic/595 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:31.438 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/595 2024-05-04T18:51:31.438 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/595 2024-05-04T18:51:31.438 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:31.440 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:31.448 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:31.448 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/596) 2024-05-04T18:51:31.604 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:31.608 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:31.608 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:31.713 INFO:teuthology.orchestra.run.smithi052.stdout:generic/596 [not run] accton utility required, skipped this test 2024-05-04T18:51:31.717 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/596 2024-05-04T18:51:31.717 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/596 2024-05-04T18:51:31.717 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:31.719 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:31.727 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:31.727 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/597) 2024-05-04T18:51:31.880 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:31.884 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:31.884 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:32.356 INFO:teuthology.orchestra.run.smithi052.stdout:generic/597 - output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/597.out.bad) 2024-05-04T18:51:32.358 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/597.out 2024-05-04 12:19:32.000992404 +0000 2024-05-04T18:51:32.358 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/597.out.bad 2024-05-04 18:51:32.351179132 +0000 2024-05-04T18:51:32.358 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -4,11 +4,11 @@ 2024-05-04T18:51:32.358 INFO:teuthology.orchestra.run.smithi052.stdout: fs.protected_symlinks = 0 2024-05-04T18:51:32.358 INFO:teuthology.orchestra.run.smithi052.stdout: successfully followed symlink 2024-05-04T18:51:32.358 INFO:teuthology.orchestra.run.smithi052.stdout: fs.protected_symlinks = 1 2024-05-04T18:51:32.359 INFO:teuthology.orchestra.run.smithi052.stdout: -Permission denied 2024-05-04T18:51:32.359 INFO:teuthology.orchestra.run.smithi052.stdout: +successfully followed symlink 2024-05-04T18:51:32.359 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:32.359 INFO:teuthology.orchestra.run.smithi052.stdout: == Test hardlink create protection when 2024-05-04T18:51:32.359 INFO:teuthology.orchestra.run.smithi052.stdout: == process != target owner and process cannot read target 2024-05-04T18:51:32.359 INFO:teuthology.orchestra.run.smithi052.stdout: fs.protected_hardlinks = 0 2024-05-04T18:51:32.359 INFO:teuthology.orchestra.run.smithi052.stdout: -successfully created hardlink 2024-05-04T18:51:32.359 INFO:teuthology.orchestra.run.smithi052.stdout: +Permission denied 2024-05-04T18:51:32.359 INFO:teuthology.orchestra.run.smithi052.stdout: fs.protected_hardlinks = 1 2024-05-04T18:51:32.359 INFO:teuthology.orchestra.run.smithi052.stdout: -ln: failed to create hard link 'TEST_DIR/597/sticky_dir/hardlink' => 'TEST_DIR/597/target': Operation not permitted 2024-05-04T18:51:32.359 INFO:teuthology.orchestra.run.smithi052.stdout: +Permission denied 2024-05-04T18:51:32.360 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/597 2024-05-04T18:51:32.360 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/597 2024-05-04T18:51:32.361 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T18:51:32.363 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:32.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T18:51:32.371 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T18:51:32.371 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T18:51:32.371 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/598) 2024-05-04T18:51:32.873 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:32.877 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:32.878 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:33.339 INFO:teuthology.orchestra.run.smithi052.stdout:generic/598 - output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/598.out.bad) 2024-05-04T18:51:33.341 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/598.out 2024-05-04 12:19:32.000992404 +0000 2024-05-04T18:51:33.341 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/598.out.bad 2024-05-04 18:51:33.335166558 +0000 2024-05-04T18:51:33.341 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -5,13 +5,10 @@ 2024-05-04T18:51:33.341 INFO:teuthology.orchestra.run.smithi052.stdout: = only group writable dir 2024-05-04T18:51:33.341 INFO:teuthology.orchestra.run.smithi052.stdout: fs.protected_regular = 1 2024-05-04T18:51:33.341 INFO:teuthology.orchestra.run.smithi052.stdout: = group & world writable dir 2024-05-04T18:51:33.341 INFO:teuthology.orchestra.run.smithi052.stdout: -Permission denied 2024-05-04T18:51:33.341 INFO:teuthology.orchestra.run.smithi052.stdout: = only group writable dir 2024-05-04T18:51:33.341 INFO:teuthology.orchestra.run.smithi052.stdout: fs.protected_regular = 2 2024-05-04T18:51:33.341 INFO:teuthology.orchestra.run.smithi052.stdout: = group & world writable dir 2024-05-04T18:51:33.341 INFO:teuthology.orchestra.run.smithi052.stdout: -Permission denied 2024-05-04T18:51:33.341 INFO:teuthology.orchestra.run.smithi052.stdout: = only group writable dir 2024-05-04T18:51:33.341 INFO:teuthology.orchestra.run.smithi052.stdout: -Permission denied 2024-05-04T18:51:33.342 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:33.342 INFO:teuthology.orchestra.run.smithi052.stdout: == Test fifo open when owned by another and fifo owner != dir owner 2024-05-04T18:51:33.342 INFO:teuthology.orchestra.run.smithi052.stdout: fs.protected_fifos = 0 2024-05-04T18:51:33.342 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -19,10 +16,7 @@ 2024-05-04T18:51:33.342 INFO:teuthology.orchestra.run.smithi052.stdout: = only group writable dir 2024-05-04T18:51:33.342 INFO:teuthology.orchestra.run.smithi052.stdout: fs.protected_fifos = 1 2024-05-04T18:51:33.342 INFO:teuthology.orchestra.run.smithi052.stdout: = group & world writable dir 2024-05-04T18:51:33.342 INFO:teuthology.orchestra.run.smithi052.stdout: -Permission denied 2024-05-04T18:51:33.342 INFO:teuthology.orchestra.run.smithi052.stdout: = only group writable dir 2024-05-04T18:51:33.342 INFO:teuthology.orchestra.run.smithi052.stdout: fs.protected_fifos = 2 2024-05-04T18:51:33.342 INFO:teuthology.orchestra.run.smithi052.stdout: = group & world writable dir 2024-05-04T18:51:33.342 INFO:teuthology.orchestra.run.smithi052.stdout: -Permission denied 2024-05-04T18:51:33.342 INFO:teuthology.orchestra.run.smithi052.stdout: = only group writable dir 2024-05-04T18:51:33.342 INFO:teuthology.orchestra.run.smithi052.stdout: -Permission denied 2024-05-04T18:51:33.343 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/598 2024-05-04T18:51:33.343 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/598 2024-05-04T18:51:33.343 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T18:51:33.345 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:33.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T18:51:33.355 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T18:51:33.355 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T18:51:33.355 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/599) 2024-05-04T18:51:33.856 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:33.860 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:33.860 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:33.966 INFO:teuthology.orchestra.run.smithi052.stdout:generic/599 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:33.969 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/599 2024-05-04T18:51:33.969 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/599 2024-05-04T18:51:33.969 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:33.971 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:33.980 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:33.980 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/600) 2024-05-04T18:51:34.132 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:34.136 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:34.136 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:34.241 INFO:teuthology.orchestra.run.smithi052.stdout:generic/600 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:34.245 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/600 2024-05-04T18:51:34.245 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/600 2024-05-04T18:51:34.245 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:34.247 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:34.255 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:34.256 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/601) 2024-05-04T18:51:34.404 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:34.410 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:34.410 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:34.520 INFO:teuthology.orchestra.run.smithi052.stdout:generic/601 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:34.524 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/601 2024-05-04T18:51:34.524 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/601 2024-05-04T18:51:34.524 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:34.526 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:34.534 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:34.534 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/602) 2024-05-04T18:51:34.685 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:34.689 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:34.690 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:34.822 INFO:teuthology.orchestra.run.smithi052.stdout:generic/602 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:34.825 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/602 2024-05-04T18:51:34.826 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/602 2024-05-04T18:51:34.826 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:34.828 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:34.836 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:34.836 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/603) 2024-05-04T18:51:34.990 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:34.994 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:34.994 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:35.098 INFO:teuthology.orchestra.run.smithi052.stdout:generic/603 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:35.102 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/603 2024-05-04T18:51:35.102 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/603 2024-05-04T18:51:35.102 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:35.104 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:35.113 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:35.113 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/604) 2024-05-04T18:51:35.263 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:35.267 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:35.267 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:35.368 INFO:teuthology.orchestra.run.smithi052.stdout:generic/604 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:35.372 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/604 2024-05-04T18:51:35.372 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/604 2024-05-04T18:51:35.372 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:35.374 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:35.382 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:35.382 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/605) 2024-05-04T18:51:35.535 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:35.539 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:35.539 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:35.641 INFO:teuthology.orchestra.run.smithi052.stdout:generic/605 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:35.644 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/605 2024-05-04T18:51:35.644 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/605 2024-05-04T18:51:35.644 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:35.647 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:35.654 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:35.655 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/606) 2024-05-04T18:51:35.804 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:35.808 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:35.809 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:35.910 INFO:teuthology.orchestra.run.smithi052.stdout:generic/606 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:35.914 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/606 2024-05-04T18:51:35.914 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/606 2024-05-04T18:51:35.914 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:35.916 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:35.924 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:35.924 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/607) 2024-05-04T18:51:36.076 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:36.080 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:36.080 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:36.181 INFO:teuthology.orchestra.run.smithi052.stdout:generic/607 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:36.185 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/607 2024-05-04T18:51:36.185 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/607 2024-05-04T18:51:36.185 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:36.187 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:36.195 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:36.196 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/608) 2024-05-04T18:51:36.344 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:36.348 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:36.348 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:36.448 INFO:teuthology.orchestra.run.smithi052.stdout:generic/608 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:36.452 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/608 2024-05-04T18:51:36.452 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/608 2024-05-04T18:51:36.452 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:36.454 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:36.461 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:36.462 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/609) 2024-05-04T18:51:36.613 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:36.617 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:36.617 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:37.241 INFO:teuthology.orchestra.run.smithi052.stdout:generic/609 1s 2024-05-04T18:51:37.249 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/609 2024-05-04T18:51:37.249 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:37.253 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:37.370 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:37.370 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/610) 2024-05-04T18:51:37.784 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:37.789 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:37.789 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:37.890 INFO:teuthology.orchestra.run.smithi052.stdout:generic/610 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:37.893 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/610 2024-05-04T18:51:37.893 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/610 2024-05-04T18:51:37.894 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:37.896 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:37.904 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:37.905 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/611) 2024-05-04T18:51:38.057 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:38.058 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:38.058 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:38.161 INFO:teuthology.orchestra.run.smithi052.stdout:generic/611 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:38.164 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/611 2024-05-04T18:51:38.164 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/611 2024-05-04T18:51:38.164 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:38.167 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:38.175 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:38.175 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/612) 2024-05-04T18:51:38.330 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:38.335 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:38.335 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:38.497 INFO:teuthology.orchestra.run.smithi052.stdout:generic/612 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T18:51:38.501 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/612 2024-05-04T18:51:38.501 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/612 2024-05-04T18:51:38.501 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:38.503 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:38.511 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:38.511 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/613) 2024-05-04T18:51:39.009 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:39.011 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:39.011 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:39.140 INFO:teuthology.orchestra.run.smithi052.stdout:generic/613 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:39.144 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/613 2024-05-04T18:51:39.144 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/613 2024-05-04T18:51:39.144 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:39.146 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:39.154 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:39.154 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/614) 2024-05-04T18:51:39.316 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:39.320 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:39.320 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:39.422 INFO:teuthology.orchestra.run.smithi052.stdout:generic/614 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:39.425 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/614 2024-05-04T18:51:39.426 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/614 2024-05-04T18:51:39.426 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:39.428 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:39.436 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:39.436 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/615) 2024-05-04T18:51:39.591 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:39.595 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:39.595 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:39.698 INFO:teuthology.orchestra.run.smithi052.stdout:generic/615 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T18:51:39.702 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/615 2024-05-04T18:51:39.702 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/615 2024-05-04T18:51:39.702 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:51:39.704 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:39.712 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:51:39.712 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/616) 2024-05-04T18:51:39.861 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:51:39.865 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:51:39.865 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:51:53.087 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:51:53.124 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:52:23.376 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:52:23.379 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:52:53.424 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:52:53.427 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:53:23.513 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:53:23.516 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:53:53.560 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:53:53.563 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:54:23.722 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:54:23.725 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:54:53.856 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:54:53.859 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:55:23.927 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:55:23.930 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:55:53.976 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:55:53.979 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:56:24.057 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:56:24.060 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:56:54.104 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:56:54.107 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:57:24.201 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:57:24.204 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:57:54.323 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:57:54.326 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:58:24.397 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:58:24.400 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:58:54.443 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:58:54.446 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:59:22.655 INFO:teuthology.orchestra.run.smithi052.stdout:generic/616 463s 2024-05-04T18:59:22.663 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/616 2024-05-04T18:59:22.663 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T18:59:22.667 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:59:22.718 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T18:59:22.719 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/617) 2024-05-04T18:59:22.920 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T18:59:22.924 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T18:59:22.924 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T18:59:24.499 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:59:24.501 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:59:54.622 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T18:59:54.625 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:00:24.673 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:00:24.676 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:00:54.800 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:00:54.803 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:01:24.846 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:01:24.849 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:01:48.094 INFO:teuthology.orchestra.run.smithi052.stdout:generic/617 146s 2024-05-04T19:01:48.102 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/617 2024-05-04T19:01:48.102 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:48.106 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:48.118 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:48.119 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/618) 2024-05-04T19:01:48.659 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:48.663 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:48.663 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:48.766 INFO:teuthology.orchestra.run.smithi052.stdout:generic/618 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:01:48.770 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/618 2024-05-04T19:01:48.770 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/618 2024-05-04T19:01:48.770 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:48.772 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:48.780 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:48.780 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/619) 2024-05-04T19:01:48.934 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:48.938 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:48.938 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:49.040 INFO:teuthology.orchestra.run.smithi052.stdout:generic/619 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:01:49.044 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/619 2024-05-04T19:01:49.044 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/619 2024-05-04T19:01:49.044 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:49.046 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:49.054 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:49.054 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/620) 2024-05-04T19:01:49.211 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:49.215 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:49.215 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:49.368 INFO:teuthology.orchestra.run.smithi052.stdout:generic/620 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:01:49.371 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/620 2024-05-04T19:01:49.371 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/620 2024-05-04T19:01:49.371 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:49.374 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:49.382 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:49.382 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/621) 2024-05-04T19:01:49.535 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:49.539 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:49.539 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:49.670 INFO:teuthology.orchestra.run.smithi052.stdout:generic/621 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:01:49.674 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/621 2024-05-04T19:01:49.674 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/621 2024-05-04T19:01:49.674 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:49.676 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:49.684 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:49.684 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/622) 2024-05-04T19:01:49.834 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:49.838 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:49.838 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:49.943 INFO:teuthology.orchestra.run.smithi052.stdout:generic/622 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:01:49.946 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/622 2024-05-04T19:01:49.946 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/622 2024-05-04T19:01:49.946 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:49.948 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:49.956 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:49.956 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/623) 2024-05-04T19:01:50.114 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:50.119 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:50.119 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:50.222 INFO:teuthology.orchestra.run.smithi052.stdout:generic/623 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:01:50.226 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/623 2024-05-04T19:01:50.226 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/623 2024-05-04T19:01:50.226 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:50.228 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:50.236 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:50.237 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/624) 2024-05-04T19:01:50.392 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:50.397 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:50.397 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:50.501 INFO:teuthology.orchestra.run.smithi052.stdout:generic/624 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:01:50.504 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/624 2024-05-04T19:01:50.504 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/624 2024-05-04T19:01:50.504 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:50.507 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:50.514 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:50.515 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/625) 2024-05-04T19:01:50.667 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:50.671 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:50.671 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:50.777 INFO:teuthology.orchestra.run.smithi052.stdout:generic/625 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:01:50.781 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/625 2024-05-04T19:01:50.781 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/625 2024-05-04T19:01:50.781 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:50.783 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:50.791 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:50.792 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/626) 2024-05-04T19:01:50.948 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:50.952 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:50.952 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:51.058 INFO:teuthology.orchestra.run.smithi052.stdout:generic/626 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:01:51.062 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/626 2024-05-04T19:01:51.062 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/626 2024-05-04T19:01:51.062 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:51.064 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:51.072 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:51.072 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/627) 2024-05-04T19:01:51.226 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:51.230 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:51.230 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:51.336 INFO:teuthology.orchestra.run.smithi052.stdout:generic/627 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:01:51.339 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/627 2024-05-04T19:01:51.339 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/627 2024-05-04T19:01:51.339 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:51.342 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:51.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:51.349 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/628) 2024-05-04T19:01:51.500 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:51.504 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:51.504 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:51.631 INFO:teuthology.orchestra.run.smithi052.stdout:generic/628 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:01:51.634 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/628 2024-05-04T19:01:51.634 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/628 2024-05-04T19:01:51.634 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:51.636 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:51.644 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:51.644 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/629) 2024-05-04T19:01:51.798 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:51.803 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:51.803 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:generic/629 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:01:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/629 2024-05-04T19:01:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/629 2024-05-04T19:01:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:51.937 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:51.945 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:51.946 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/630) 2024-05-04T19:01:52.099 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:52.103 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:52.103 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:52.204 INFO:teuthology.orchestra.run.smithi052.stdout:generic/630 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:01:52.208 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/630 2024-05-04T19:01:52.208 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/630 2024-05-04T19:01:52.208 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:52.210 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:52.218 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:52.218 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/631) 2024-05-04T19:01:52.372 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:52.376 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:52.376 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:52.481 INFO:teuthology.orchestra.run.smithi052.stdout:generic/631 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:01:52.485 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/631 2024-05-04T19:01:52.485 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/631 2024-05-04T19:01:52.485 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:01:52.487 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:52.495 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:01:52.495 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/632) 2024-05-04T19:01:52.649 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:01:52.653 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:01:52.653 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:01:54.902 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:01:54.907 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:02:00.182 INFO:teuthology.orchestra.run.smithi052.stdout:generic/632 8s 2024-05-04T19:02:00.190 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/632 2024-05-04T19:02:00.190 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:00.246 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:00.246 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/633) 2024-05-04T19:02:00.446 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:00.451 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:00.929 INFO:teuthology.orchestra.run.smithi052.stdout:generic/633 [failed, exit status 1]- output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/633.out.bad) 2024-05-04T19:02:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/633.out 2024-05-04 12:19:32.004992343 +0000 2024-05-04T19:02:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/633.out.bad 2024-05-04 19:02:00.923152713 +0000 2024-05-04T19:02:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -1,2 +1,5 @@ 2024-05-04T19:02:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: QA output created by 633 2024-05-04T19:02:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: Silence is golden 2024-05-04T19:02:00.932 INFO:teuthology.orchestra.run.smithi052.stdout: +vfstest.c: 1164: protected_symlinks - Success - failure: openat 2024-05-04T19:02:00.932 INFO:teuthology.orchestra.run.smithi052.stdout: +vfstest.c: 1171: protected_symlinks - Success - failure: wait_for_pid 2024-05-04T19:02:00.932 INFO:teuthology.orchestra.run.smithi052.stdout: +vfstest.c: 2418: run_test - Success - failure: following protected symlinks on regular mounts 2024-05-04T19:02:00.934 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/633 2024-05-04T19:02:00.934 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/633 2024-05-04T19:02:00.934 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T19:02:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:01.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T19:02:01.088 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T19:02:01.088 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T19:02:01.089 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/634) 2024-05-04T19:02:01.458 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:01.462 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:01.462 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:01.562 INFO:teuthology.orchestra.run.smithi052.stdout:generic/634 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:02:01.565 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/634 2024-05-04T19:02:01.565 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/634 2024-05-04T19:02:01.565 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:01.567 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:01.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:01.575 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/635) 2024-05-04T19:02:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:01.732 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:01.732 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:01.839 INFO:teuthology.orchestra.run.smithi052.stdout:generic/635 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:02:01.842 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/635 2024-05-04T19:02:01.842 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/635 2024-05-04T19:02:01.842 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:01.844 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:01.852 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:01.852 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/636) 2024-05-04T19:02:02.003 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:02.007 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:02.007 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:02.108 INFO:teuthology.orchestra.run.smithi052.stdout:generic/636 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:02:02.112 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/636 2024-05-04T19:02:02.113 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/636 2024-05-04T19:02:02.113 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:02.115 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:02.123 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:02.123 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/637) 2024-05-04T19:02:02.281 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:02.285 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:02.285 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:04.434 INFO:teuthology.orchestra.run.smithi052.stdout:generic/637 2s 2024-05-04T19:02:04.441 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/637 2024-05-04T19:02:04.442 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:04.445 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:04.459 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:04.459 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/638) 2024-05-04T19:02:05.026 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:05.030 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:05.030 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:05.192 INFO:teuthology.orchestra.run.smithi052.stdout:generic/638 0s 2024-05-04T19:02:05.197 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/638 2024-05-04T19:02:05.197 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:05.199 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:05.210 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:05.211 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/639) 2024-05-04T19:02:05.365 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:05.369 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:05.370 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:06.686 INFO:teuthology.orchestra.run.smithi052.stdout:generic/639 1s 2024-05-04T19:02:06.694 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/639 2024-05-04T19:02:06.694 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:06.698 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:06.709 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:06.710 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/640) 2024-05-04T19:02:06.916 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:06.921 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:06.921 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:07.069 INFO:teuthology.orchestra.run.smithi052.stdout:generic/640 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:02:07.073 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/640 2024-05-04T19:02:07.073 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/640 2024-05-04T19:02:07.074 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:07.077 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:07.087 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:07.088 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/641) 2024-05-04T19:02:07.269 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:07.274 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:07.274 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:07.397 INFO:teuthology.orchestra.run.smithi052.stdout:generic/641 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:02:07.401 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/641 2024-05-04T19:02:07.401 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/641 2024-05-04T19:02:07.401 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:07.404 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:07.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:07.413 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/642) 2024-05-04T19:02:07.596 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:07.602 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:07.602 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:07.731 INFO:teuthology.orchestra.run.smithi052.stdout:generic/642 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:02:07.736 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/642 2024-05-04T19:02:07.736 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/642 2024-05-04T19:02:07.736 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:07.739 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:07.749 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:07.749 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/643) 2024-05-04T19:02:07.932 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:07.937 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:07.937 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:08.051 INFO:teuthology.orchestra.run.smithi052.stdout:generic/643 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:02:08.055 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/643 2024-05-04T19:02:08.055 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/643 2024-05-04T19:02:08.055 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:08.057 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:08.066 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:08.067 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/644) 2024-05-04T19:02:08.220 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:08.225 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:08.225 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:08.331 INFO:teuthology.orchestra.run.smithi052.stdout:generic/644 [not run] vfstest not support by ceph-fuse 2024-05-04T19:02:08.334 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/644 2024-05-04T19:02:08.335 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/644 2024-05-04T19:02:08.335 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:08.337 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:08.344 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:08.345 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/645) 2024-05-04T19:02:08.501 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:08.505 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:08.505 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:08.613 INFO:teuthology.orchestra.run.smithi052.stdout:generic/645 [not run] vfstest not support by ceph-fuse 2024-05-04T19:02:08.617 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/645 2024-05-04T19:02:08.618 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/645 2024-05-04T19:02:08.618 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:08.620 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:08.628 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:08.628 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/646) 2024-05-04T19:02:08.785 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:08.789 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:08.789 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:08.890 INFO:teuthology.orchestra.run.smithi052.stdout:generic/646 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:02:08.893 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/646 2024-05-04T19:02:08.893 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/646 2024-05-04T19:02:08.894 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:08.896 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:08.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:08.904 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/647) 2024-05-04T19:02:09.057 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:09.062 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:09.062 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:09.445 INFO:teuthology.orchestra.run.smithi052.stdout:generic/647 0s 2024-05-04T19:02:09.452 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/647 2024-05-04T19:02:09.452 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:09.457 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:09.464 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:09.465 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/648) 2024-05-04T19:02:09.982 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:09.986 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:09.986 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:10.107 INFO:teuthology.orchestra.run.smithi052.stdout:generic/648 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T19:02:10.111 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/648 2024-05-04T19:02:10.111 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/648 2024-05-04T19:02:10.111 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:10.121 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:10.122 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/649) 2024-05-04T19:02:10.272 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:10.276 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:10.276 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:10.467 INFO:teuthology.orchestra.run.smithi052.stdout:generic/649 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T19:02:10.470 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/649 2024-05-04T19:02:10.470 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/649 2024-05-04T19:02:10.470 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T19:02:10.472 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:10.480 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T19:02:10.481 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/650) 2024-05-04T19:02:10.991 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T19:02:10.995 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T19:02:10.995 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T19:02:24.983 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:02:24.986 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:02:55.248 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:02:55.250 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:03:25.474 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:03:25.476 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:03:56.054 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:03:56.057 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:04:26.141 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:04:26.143 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:04:56.361 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:04:56.364 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:05:26.818 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:05:26.821 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:05:57.006 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:05:57.008 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:06:27.168 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:06:27.170 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:06:57.472 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:06:57.478 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:07:27.532 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:07:27.534 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:07:57.707 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:07:57.710 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:07:57.772 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-04T19:07:57.770+0000 7f6f26d7f640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 443892) UID: 0 2024-05-04T19:07:57.776 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T19:07:57.770+0000 7fbf91e1b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 443892) UID: 0 2024-05-04T19:16:29.029 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:16:29.032 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:16:59.329 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:16:59.331 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:17:29.393 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:17:29.396 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:17:59.584 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:17:59.587 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:18:29.848 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:18:29.852 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:19:00.367 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:19:00.372 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:19:30.525 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:19:30.528 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:20:00.732 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:20:00.738 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:20:31.051 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:20:31.053 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:21:01.297 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:21:01.300 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:21:32.020 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:21:32.022 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:22:02.368 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:22:02.370 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:22:32.534 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:22:32.536 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:23:02.615 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:23:02.618 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:23:32.719 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:23:32.722 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:24:02.890 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:24:02.892 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:24:32.931 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:24:32.934 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:24:32.991 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T19:24:32.984+0000 7fb74db8d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36769) UID: 0 2024-05-04T19:24:32.992 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T19:24:32.984+0000 7f1412d90640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36769) UID: 0 2024-05-04T19:24:32.992 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T19:24:32.984+0000 7f81dc32b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36769) UID: 0 2024-05-04T19:24:32.992 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T19:24:32.984+0000 7fc53dde6640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36769) UID: 0 2024-05-04T19:27:33.624 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:27:33.627 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:28:03.826 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:28:03.828 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:28:03.870 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-04T19:28:03.867+0000 7f6f26d7f640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 446565) UID: 0 2024-05-04T19:28:03.870 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T19:28:03.867+0000 7fbf91e1b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 446565) UID: 0 2024-05-04T19:35:57.101 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:35:57.104 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:36:28.418 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:36:28.421 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:36:28.497 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T19:36:28.488+0000 7f7b9124a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 447717) UID: 0 2024-05-04T19:36:28.497 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T19:36:28.488+0000 7f3c99d9c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 447717) UID: 0 2024-05-04T19:36:28.497 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T19:36:28.488+0000 7fd3bf2df640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 447717) UID: 0 2024-05-04T19:36:28.501 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T19:36:28.496+0000 7f17989d7640 -1 Fail to open '/proc/447717/cmdline' error = (2) No such file or directory 2024-05-04T19:36:28.501 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T19:36:28.496+0000 7f17989d7640 -1 received signal: Hangup from (PID: 447717) UID: 0 2024-05-04T19:44:52.412 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:44:52.414 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:44:52.464 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T19:44:52.459+0000 7f1412d90640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36859) UID: 0 2024-05-04T19:44:52.464 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T19:44:52.459+0000 7fc53dde6640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36859) UID: 0 2024-05-04T19:44:52.465 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T19:44:52.459+0000 7fb74db8d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36859) UID: 0 2024-05-04T19:44:52.465 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T19:44:52.459+0000 7f81dc32b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36859) UID: 0 2024-05-04T19:50:32.229 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:50:32.232 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T19:50:32.410 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-04T19:50:32.406+0000 7f6f26d7f640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 449600) UID: 0 2024-05-04T19:50:32.411 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T19:50:32.406+0000 7fbf91e1b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 449600) UID: 0 2024-05-04T19:59:56.862 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T19:59:56.858+0000 7f7b9124a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 450704) UID: 0 2024-05-04T19:59:56.863 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T19:59:56.858+0000 7f3c99d9c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 450704) UID: 0 2024-05-04T19:59:56.863 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T19:59:56.858+0000 7fd3bf2df640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 450704) UID: 0 2024-05-04T19:59:56.864 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T19:59:56.858+0000 7f17989d7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 450704) UID: 0 2024-05-04T20:01:50.683 INFO:teuthology.orchestra.run.smithi052.stdout:generic/650 3579s 2024-05-04T20:01:51.055 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/650 2024-05-04T20:01:51.055 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:01:51.058 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:51.090 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:01:51.090 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/651) 2024-05-04T20:01:52.484 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:01:52.489 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:01:52.489 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:52.824 INFO:teuthology.orchestra.run.smithi052.stdout:generic/651 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:01:52.828 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/651 2024-05-04T20:01:52.828 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/651 2024-05-04T20:01:52.828 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:01:52.830 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:52.840 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:01:52.841 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/652) 2024-05-04T20:01:53.135 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:01:53.142 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:01:53.142 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:53.306 INFO:teuthology.orchestra.run.smithi052.stdout:generic/652 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:01:53.311 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/652 2024-05-04T20:01:53.311 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/652 2024-05-04T20:01:53.311 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:01:53.314 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:53.329 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:01:53.329 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/653) 2024-05-04T20:01:53.646 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:01:53.653 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:01:53.653 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:53.832 INFO:teuthology.orchestra.run.smithi052.stdout:generic/653 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:01:53.837 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/653 2024-05-04T20:01:53.837 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/653 2024-05-04T20:01:53.837 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:01:53.839 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:53.849 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:01:53.849 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/654) 2024-05-04T20:01:54.153 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:01:54.158 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:01:54.158 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:54.324 INFO:teuthology.orchestra.run.smithi052.stdout:generic/654 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:01:54.333 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/654 2024-05-04T20:01:54.333 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/654 2024-05-04T20:01:54.333 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:01:54.335 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:54.355 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:01:54.355 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/655) 2024-05-04T20:01:54.658 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:01:54.664 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:01:54.664 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:54.821 INFO:teuthology.orchestra.run.smithi052.stdout:generic/655 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:01:54.824 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/655 2024-05-04T20:01:54.824 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/655 2024-05-04T20:01:54.825 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:01:54.827 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:54.841 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:01:54.842 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/656) 2024-05-04T20:01:55.159 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:01:55.165 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:01:55.165 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:55.560 INFO:teuthology.orchestra.run.smithi052.stdout:generic/656 [not run] vfstest not support by ceph-fuse 2024-05-04T20:01:55.565 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/656 2024-05-04T20:01:55.565 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/656 2024-05-04T20:01:55.565 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:01:55.567 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:55.576 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:01:55.577 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/657) 2024-05-04T20:01:55.864 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:01:55.870 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:01:55.870 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:56.038 INFO:teuthology.orchestra.run.smithi052.stdout:generic/657 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:01:56.041 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/657 2024-05-04T20:01:56.042 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/657 2024-05-04T20:01:56.042 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:01:56.044 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:56.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:01:56.053 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/658) 2024-05-04T20:01:56.411 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:01:56.417 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:01:56.417 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:56.663 INFO:teuthology.orchestra.run.smithi052.stdout:generic/658 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:01:56.667 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/658 2024-05-04T20:01:56.667 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/658 2024-05-04T20:01:56.667 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:01:56.669 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:56.685 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:01:56.685 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/659) 2024-05-04T20:01:56.940 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:01:56.944 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:01:56.944 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:57.128 INFO:teuthology.orchestra.run.smithi052.stdout:generic/659 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:01:57.133 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/659 2024-05-04T20:01:57.133 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/659 2024-05-04T20:01:57.133 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:01:57.135 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:57.145 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:01:57.146 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/660) 2024-05-04T20:01:57.374 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:01:57.378 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:01:57.378 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:57.576 INFO:teuthology.orchestra.run.smithi052.stdout:generic/660 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:01:57.582 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/660 2024-05-04T20:01:57.582 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/660 2024-05-04T20:01:57.582 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:01:57.584 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:57.603 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:01:57.603 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/661) 2024-05-04T20:01:57.853 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:01:57.858 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:01:57.858 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:58.026 INFO:teuthology.orchestra.run.smithi052.stdout:generic/661 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:01:58.030 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/661 2024-05-04T20:01:58.030 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/661 2024-05-04T20:01:58.030 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:01:58.032 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:58.047 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:01:58.047 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/662) 2024-05-04T20:01:58.316 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:01:58.324 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:01:58.325 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:58.761 INFO:teuthology.orchestra.run.smithi052.stdout:generic/662 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:01:58.765 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/662 2024-05-04T20:01:58.765 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/662 2024-05-04T20:01:58.766 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:01:58.768 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:58.777 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:01:58.778 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/663) 2024-05-04T20:01:59.037 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:01:59.043 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:01:59.043 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:59.265 INFO:teuthology.orchestra.run.smithi052.stdout:generic/663 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:01:59.269 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/663 2024-05-04T20:01:59.269 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/663 2024-05-04T20:01:59.269 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:01:59.272 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:01:59.296 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:01:59.296 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/664) 2024-05-04T20:01:59.594 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:01:59.599 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:01:59.600 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:00.079 INFO:teuthology.orchestra.run.smithi052.stdout:generic/664 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:02:00.084 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/664 2024-05-04T20:02:00.084 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/664 2024-05-04T20:02:00.084 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:02:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:00.096 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:02:00.096 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/665) 2024-05-04T20:02:00.355 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:02:00.361 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:02:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:00.532 INFO:teuthology.orchestra.run.smithi052.stdout:generic/665 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:02:00.544 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/665 2024-05-04T20:02:00.544 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/665 2024-05-04T20:02:00.544 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:02:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:00.559 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:02:00.559 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/666) 2024-05-04T20:02:00.830 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:02:00.839 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:02:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:01.042 INFO:teuthology.orchestra.run.smithi052.stdout:generic/666 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:02:01.046 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/666 2024-05-04T20:02:01.046 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/666 2024-05-04T20:02:01.046 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:02:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:01.078 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:02:01.078 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/667) 2024-05-04T20:02:01.363 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:02:01.368 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:02:01.368 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:01.850 INFO:teuthology.orchestra.run.smithi052.stdout:generic/667 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:02:01.855 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/667 2024-05-04T20:02:01.855 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/667 2024-05-04T20:02:01.855 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:02:01.858 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:01.870 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:02:01.870 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/668) 2024-05-04T20:02:02.159 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:02:02.163 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:02:02.164 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:02.337 INFO:teuthology.orchestra.run.smithi052.stdout:generic/668 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:02:02.343 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/668 2024-05-04T20:02:02.343 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/668 2024-05-04T20:02:02.343 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:02:02.346 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:02.363 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:02:02.363 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/669) 2024-05-04T20:02:02.698 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:02:02.703 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:02:02.703 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:02.867 INFO:teuthology.orchestra.run.smithi052.stdout:generic/669 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:02:02.871 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/669 2024-05-04T20:02:02.871 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/669 2024-05-04T20:02:02.871 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:02:02.874 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:02.889 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:02:02.889 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/670) 2024-05-04T20:02:03.125 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:02:03.129 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:02:03.129 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:03.280 INFO:teuthology.orchestra.run.smithi052.stdout:generic/670 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:02:03.285 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/670 2024-05-04T20:02:03.285 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/670 2024-05-04T20:02:03.285 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:02:03.287 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:03.296 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:02:03.296 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/671) 2024-05-04T20:02:03.614 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:02:03.619 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:02:03.619 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:03.810 INFO:teuthology.orchestra.run.smithi052.stdout:generic/671 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:02:03.815 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/671 2024-05-04T20:02:03.815 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/671 2024-05-04T20:02:03.816 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:02:03.818 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:03.830 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:02:03.830 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/672) 2024-05-04T20:02:04.118 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:02:04.123 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:02:04.123 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:04.284 INFO:teuthology.orchestra.run.smithi052.stdout:generic/672 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:02:04.288 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/672 2024-05-04T20:02:04.288 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/672 2024-05-04T20:02:04.288 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:02:04.291 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:04.320 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:02:04.320 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/673) 2024-05-04T20:02:04.576 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:02:04.581 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:02:04.581 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:06.041 INFO:teuthology.orchestra.run.smithi052.stdout:generic/673 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:02:06.052 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/673 2024-05-04T20:02:06.052 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/673 2024-05-04T20:02:06.053 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:02:06.055 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:06.067 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:02:06.067 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/674) 2024-05-04T20:02:06.349 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:02:06.358 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:02:06.358 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:06.579 INFO:teuthology.orchestra.run.smithi052.stdout:generic/674 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:02:06.583 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/674 2024-05-04T20:02:06.584 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/674 2024-05-04T20:02:06.584 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:02:06.586 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:06.600 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:02:06.600 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/675) 2024-05-04T20:02:06.895 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:02:06.902 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:02:06.902 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:07.280 INFO:teuthology.orchestra.run.smithi052.stdout:generic/675 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:02:07.294 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/675 2024-05-04T20:02:07.294 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/675 2024-05-04T20:02:07.294 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:02:07.301 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:02:07.317 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:02:07.317 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/676) 2024-05-04T20:02:07.629 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:02:07.635 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:02:07.636 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:11:16.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:11:16.508 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:11:16.626 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T20:11:16.619+0000 7fb74db8d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36943) UID: 0 2024-05-04T20:11:16.626 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T20:11:16.619+0000 7f1412d90640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36943) UID: 0 2024-05-04T20:11:16.626 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T20:11:16.619+0000 7f81dc32b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36943) UID: 0 2024-05-04T20:11:16.627 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T20:11:16.619+0000 7fc53dde6640 -1 Fail to open '/proc/36943/cmdline' error = (2) No such file or directory 2024-05-04T20:11:16.627 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T20:11:16.619+0000 7fc53dde6640 -1 received signal: Hangup from (PID: 36943) UID: 0 2024-05-04T20:11:16.796 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-04T20:11:16.794+0000 7f6f26d7f640 -1 Fail to open '/proc/461377/cmdline' error = (2) No such file or directory 2024-05-04T20:11:16.797 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-04T20:11:16.794+0000 7f6f26d7f640 -1 received signal: Hangup from (PID: 461377) UID: 0 2024-05-04T20:11:16.798 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T20:11:16.794+0000 7fbf91e1b640 -1 Fail to open '/proc/461377/cmdline' error = (2) No such file or directory 2024-05-04T20:11:16.798 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T20:11:16.794+0000 7fbf91e1b640 -1 received signal: Hangup from (PID: 461377) UID: 0 2024-05-04T20:24:13.184 INFO:teuthology.orchestra.run.smithi052.stderr:error: Compressing program wrote following message to stderr when compressing log /var/log/ceph/ceph-mds.c.log.1: 2024-05-04T20:24:13.185 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: stdin: file size changed while zipping 2024-05-04T20:24:15.029 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T20:24:15.024+0000 7f17989d7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 461405) UID: 0 2024-05-04T20:24:15.030 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T20:24:15.024+0000 7f3c99d9c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 461405) UID: 0 2024-05-04T20:24:15.030 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T20:24:15.024+0000 7fd3bf2df640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 461405) UID: 0 2024-05-04T20:24:15.030 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T20:24:15.024+0000 7f7b9124a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 461405) UID: 0 2024-05-04T20:31:22.818 INFO:teuthology.orchestra.run.smithi052.stdout:generic/676 1755s 2024-05-04T20:31:22.877 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/676 2024-05-04T20:31:22.877 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:22.879 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:22.900 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:22.900 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/677) 2024-05-04T20:31:24.027 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:24.033 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:24.033 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:24.256 INFO:teuthology.orchestra.run.smithi052.stdout:generic/677 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:24.261 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/677 2024-05-04T20:31:24.261 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/677 2024-05-04T20:31:24.261 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:24.263 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:24.277 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:24.277 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/678) 2024-05-04T20:31:24.532 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:24.537 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:24.537 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:25.550 INFO:teuthology.orchestra.run.smithi052.stdout:generic/678 1s 2024-05-04T20:31:25.557 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/678 2024-05-04T20:31:25.557 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:25.563 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:25.579 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:25.580 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/679) 2024-05-04T20:31:26.237 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:26.242 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:26.242 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:26.410 INFO:teuthology.orchestra.run.smithi052.stdout:generic/679 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:26.422 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/679 2024-05-04T20:31:26.422 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/679 2024-05-04T20:31:26.422 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:26.427 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:26.444 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:26.444 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/680) 2024-05-04T20:31:26.694 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:26.699 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:26.699 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:30.137 INFO:teuthology.orchestra.run.smithi052.stdout:generic/680 4s 2024-05-04T20:31:30.148 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/680 2024-05-04T20:31:30.148 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:30.154 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:30.172 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:30.172 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/681) 2024-05-04T20:31:30.780 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:30.786 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:30.786 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:31.440 INFO:teuthology.orchestra.run.smithi052.stdout:generic/681 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-05-04T20:31:31.445 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/681 2024-05-04T20:31:31.445 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/681 2024-05-04T20:31:31.445 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:31.447 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:31.468 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:31.468 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/682) 2024-05-04T20:31:31.723 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:31.728 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:31.728 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:31.877 INFO:teuthology.orchestra.run.smithi052.stdout:generic/682 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-05-04T20:31:31.881 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/682 2024-05-04T20:31:31.881 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/682 2024-05-04T20:31:31.881 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:31.884 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:31.899 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:31.899 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/683) 2024-05-04T20:31:32.181 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:32.188 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:32.188 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:32.437 INFO:teuthology.orchestra.run.smithi052.stdout:generic/683 [not run] /tmp/tmp.FGpMFsgr0Qtest: file alloc unit 4194304 larger than op length 65536 2024-05-04T20:31:32.441 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/683 2024-05-04T20:31:32.442 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/683 2024-05-04T20:31:32.442 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:32.444 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:32.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:32.455 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/684) 2024-05-04T20:31:32.700 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:32.705 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:32.705 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:34.164 INFO:teuthology.orchestra.run.smithi052.stdout:generic/684 [not run] /tmp/tmp.FGpMFsgr0Qtest: file alloc unit 4194304 larger than op length 65536 2024-05-04T20:31:34.168 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/684 2024-05-04T20:31:34.168 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/684 2024-05-04T20:31:34.168 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:34.170 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:34.187 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:34.187 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/685) 2024-05-04T20:31:34.441 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:34.445 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:34.445 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:35.163 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:31:35.165 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:31:35.222 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-04T20:31:35.223+0000 7f6f26d7f640 -1 Fail to open '/proc/465303/cmdline' error = (2) No such file or directory 2024-05-04T20:31:35.222 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-04T20:31:35.223+0000 7f6f26d7f640 -1 received signal: Hangup from (PID: 465303) UID: 0 2024-05-04T20:31:35.223 INFO:tasks.ceph.mds.c.smithi052.stderr:2024-05-04T20:31:35.223+0000 7fbf91e1b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 465303) UID: 0 2024-05-04T20:31:35.464 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T20:31:35.463+0000 7f1412d90640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 37061) UID: 0 2024-05-04T20:31:35.464 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T20:31:35.463+0000 7fb74db8d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 37061) UID: 0 2024-05-04T20:31:35.465 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T20:31:35.463+0000 7f81dc32b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 37061) UID: 0 2024-05-04T20:31:35.465 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T20:31:35.463+0000 7fc53dde6640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 37061) UID: 0 2024-05-04T20:31:35.915 INFO:teuthology.orchestra.run.smithi052.stdout:generic/685 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-05-04T20:31:35.921 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/685 2024-05-04T20:31:35.921 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/685 2024-05-04T20:31:35.921 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:35.924 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:35.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:35.954 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/686) 2024-05-04T20:31:36.301 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:36.307 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:36.307 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:37.791 INFO:teuthology.orchestra.run.smithi052.stdout:generic/686 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-05-04T20:31:37.799 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/686 2024-05-04T20:31:37.799 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/686 2024-05-04T20:31:37.799 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:37.802 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:37.818 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:37.818 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/687) 2024-05-04T20:31:38.009 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:38.013 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:38.013 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:39.239 INFO:teuthology.orchestra.run.smithi052.stdout:generic/687 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-05-04T20:31:39.252 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/687 2024-05-04T20:31:39.252 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/687 2024-05-04T20:31:39.252 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:39.255 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:39.264 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:39.265 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/688) 2024-05-04T20:31:39.519 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:39.531 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:39.531 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:generic/688 [not run] /tmp/tmp.FGpMFsgr0Qtest: file alloc unit 4194304 larger than op length 65536 2024-05-04T20:31:39.950 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/688 2024-05-04T20:31:39.951 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/688 2024-05-04T20:31:39.951 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:39.953 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:39.970 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:39.971 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/689) 2024-05-04T20:31:40.225 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:40.229 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:40.230 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:40.632 INFO:teuthology.orchestra.run.smithi052.stdout:generic/689 [not run] vfstest not support by ceph-fuse 2024-05-04T20:31:40.637 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/689 2024-05-04T20:31:40.637 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/689 2024-05-04T20:31:40.637 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:40.642 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:40.655 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:40.655 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/690) 2024-05-04T20:31:40.881 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:40.886 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:40.886 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:41.386 INFO:teuthology.orchestra.run.smithi052.stdout:generic/690 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:41.390 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/690 2024-05-04T20:31:41.390 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/690 2024-05-04T20:31:41.390 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:41.393 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:41.411 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:41.412 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/691) 2024-05-04T20:31:41.701 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:41.706 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:41.707 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:41.872 INFO:teuthology.orchestra.run.smithi052.stdout:generic/691 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:41.876 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/691 2024-05-04T20:31:41.876 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/691 2024-05-04T20:31:41.877 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:41.880 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:41.895 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:41.895 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/692) 2024-05-04T20:31:42.147 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:42.152 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:42.152 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:42.730 INFO:teuthology.orchestra.run.smithi052.stdout:generic/692 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:42.734 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/692 2024-05-04T20:31:42.734 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/692 2024-05-04T20:31:42.734 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:42.737 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:42.756 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:42.756 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/693) 2024-05-04T20:31:43.023 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:43.028 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:43.028 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:43.436 INFO:teuthology.orchestra.run.smithi052.stdout:generic/693 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:43.440 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/693 2024-05-04T20:31:43.440 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/693 2024-05-04T20:31:43.440 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:43.443 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:43.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:43.455 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/694) 2024-05-04T20:31:43.733 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:43.738 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:43.738 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:44.590 INFO:teuthology.orchestra.run.smithi052.stdout:generic/694 1s 2024-05-04T20:31:44.601 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/694 2024-05-04T20:31:44.602 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:44.604 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:44.614 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:44.614 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/695) 2024-05-04T20:31:44.889 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:44.897 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:44.898 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:45.078 INFO:teuthology.orchestra.run.smithi052.stdout:generic/695 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:45.083 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/695 2024-05-04T20:31:45.083 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/695 2024-05-04T20:31:45.083 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:45.086 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:45.106 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:45.107 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/696) 2024-05-04T20:31:45.486 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:45.491 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:45.491 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:45.678 INFO:teuthology.orchestra.run.smithi052.stdout:generic/696 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:45.703 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/696 2024-05-04T20:31:45.703 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/696 2024-05-04T20:31:45.703 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:45.712 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:45.725 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:45.725 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/697) 2024-05-04T20:31:46.047 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:46.052 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:46.052 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:46.342 INFO:teuthology.orchestra.run.smithi052.stdout:generic/697 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-05-04T20:31:46.347 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/697 2024-05-04T20:31:46.347 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/697 2024-05-04T20:31:46.347 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:46.356 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:46.373 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:46.374 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/698) 2024-05-04T20:31:46.698 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:46.708 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:46.708 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:46.915 INFO:teuthology.orchestra.run.smithi052.stdout:generic/698 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:46.924 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/698 2024-05-04T20:31:46.924 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/698 2024-05-04T20:31:46.925 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:46.928 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:46.943 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:46.943 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/699) 2024-05-04T20:31:47.252 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:47.257 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:47.257 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:47.695 INFO:teuthology.orchestra.run.smithi052.stdout:generic/699 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:47.700 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/699 2024-05-04T20:31:47.700 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/699 2024-05-04T20:31:47.700 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:47.706 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:47.722 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:47.722 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/700) 2024-05-04T20:31:47.991 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:47.996 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:47.996 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:48.181 INFO:teuthology.orchestra.run.smithi052.stdout:generic/700 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:48.184 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/700 2024-05-04T20:31:48.185 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/700 2024-05-04T20:31:48.185 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:48.192 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:48.209 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:48.210 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/701) 2024-05-04T20:31:48.558 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:48.563 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:48.564 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:49.145 INFO:teuthology.orchestra.run.smithi052.stdout:generic/701 1s 2024-05-04T20:31:49.167 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/701 2024-05-04T20:31:49.167 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:49.169 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:49.187 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:49.187 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/702) 2024-05-04T20:31:49.913 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:49.919 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:49.919 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:50.137 INFO:teuthology.orchestra.run.smithi052.stdout:generic/702 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:50.141 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/702 2024-05-04T20:31:50.141 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/702 2024-05-04T20:31:50.141 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:50.150 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:50.166 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:50.167 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/703) 2024-05-04T20:31:50.472 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:50.477 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:50.477 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:50.687 INFO:teuthology.orchestra.run.smithi052.stdout:generic/703 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:50.692 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/703 2024-05-04T20:31:50.692 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/703 2024-05-04T20:31:50.692 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:50.697 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:50.706 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:50.707 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/704) 2024-05-04T20:31:51.064 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:51.070 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:51.070 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:51.430 INFO:teuthology.orchestra.run.smithi052.stdout:generic/704 [not run] require ceph-fuse to be valid block disk 2024-05-04T20:31:51.436 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/704 2024-05-04T20:31:51.436 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/704 2024-05-04T20:31:51.436 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:51.439 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:51.449 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:51.449 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/705) 2024-05-04T20:31:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:51.864 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:52.040 INFO:teuthology.orchestra.run.smithi052.stdout:generic/705 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:52.045 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/705 2024-05-04T20:31:52.045 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/705 2024-05-04T20:31:52.045 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:52.048 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:52.060 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:52.060 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/706) 2024-05-04T20:31:52.345 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:52.351 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:52.351 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:54.723 INFO:teuthology.orchestra.run.smithi052.stdout:generic/706 2s 2024-05-04T20:31:54.737 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/706 2024-05-04T20:31:54.737 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:54.747 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:54.758 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:54.758 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/707) 2024-05-04T20:31:55.484 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:55.488 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:55.488 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:55.649 INFO:teuthology.orchestra.run.smithi052.stdout:generic/707 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:31:55.656 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/707 2024-05-04T20:31:55.656 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/707 2024-05-04T20:31:55.656 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:55.658 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:55.667 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:55.668 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/708) 2024-05-04T20:31:55.968 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:55.973 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:55.973 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:56.697 INFO:teuthology.orchestra.run.smithi052.stdout:generic/708 1s 2024-05-04T20:31:56.705 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/708 2024-05-04T20:31:56.705 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:56.718 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:56.737 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:56.737 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/709) 2024-05-04T20:31:57.362 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:57.373 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:57.373 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:57.704 INFO:teuthology.orchestra.run.smithi052.stdout:generic/709 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-05-04T20:31:57.709 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/709 2024-05-04T20:31:57.709 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/709 2024-05-04T20:31:57.709 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:57.725 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:57.944 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:57.944 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/710) 2024-05-04T20:31:58.496 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:58.504 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:58.505 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:58.761 INFO:teuthology.orchestra.run.smithi052.stdout:generic/710 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-05-04T20:31:58.765 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/710 2024-05-04T20:31:58.765 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/710 2024-05-04T20:31:58.766 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:58.768 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:58.778 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:58.779 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/711) 2024-05-04T20:31:59.486 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:31:59.492 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:31:59.492 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:59.743 INFO:teuthology.orchestra.run.smithi052.stdout:generic/711 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-05-04T20:31:59.752 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/711 2024-05-04T20:31:59.752 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/711 2024-05-04T20:31:59.752 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:31:59.758 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:31:59.841 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:31:59.842 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/712) 2024-05-04T20:32:00.481 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:00.485 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:00.746 INFO:teuthology.orchestra.run.smithi052.stdout:generic/712 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-05-04T20:32:00.751 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/712 2024-05-04T20:32:00.751 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/712 2024-05-04T20:32:00.751 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:00.767 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:00.767 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/713) 2024-05-04T20:32:01.514 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:01.524 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:01.525 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:01.799 INFO:teuthology.orchestra.run.smithi052.stdout:generic/713 [not run] xfs_io swapext -v exchrange -s 64k -l 64k not supported on ceph-fuse 2024-05-04T20:32:01.808 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/713 2024-05-04T20:32:01.808 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/713 2024-05-04T20:32:01.808 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:01.820 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:01.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:01.832 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/714) 2024-05-04T20:32:02.567 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:02.572 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:02.572 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:02.865 INFO:teuthology.orchestra.run.smithi052.stdout:generic/714 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-05-04T20:32:02.870 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/714 2024-05-04T20:32:02.870 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/714 2024-05-04T20:32:02.870 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:02.873 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:03.049 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:03.049 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/715) 2024-05-04T20:32:03.524 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:03.531 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:03.531 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:03.810 INFO:teuthology.orchestra.run.smithi052.stdout:generic/715 [not run] xfs_io swapext -v exchrange -s 64k -l 64k not supported on ceph-fuse 2024-05-04T20:32:03.818 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/715 2024-05-04T20:32:03.818 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/715 2024-05-04T20:32:03.818 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:03.821 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:03.833 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:03.833 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/716) 2024-05-04T20:32:04.420 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:04.424 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:04.424 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:04.667 INFO:teuthology.orchestra.run.smithi052.stdout:generic/716 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-05-04T20:32:04.671 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/716 2024-05-04T20:32:04.671 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/716 2024-05-04T20:32:04.671 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:04.673 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:04.683 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:04.684 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/717) 2024-05-04T20:32:05.324 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:05.329 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:05.329 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:05.595 INFO:teuthology.orchestra.run.smithi052.stdout:generic/717 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-05-04T20:32:05.599 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/717 2024-05-04T20:32:05.599 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/717 2024-05-04T20:32:05.599 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:05.602 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:05.636 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:05.637 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/718) 2024-05-04T20:32:06.249 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:06.260 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:06.260 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:06.803 INFO:teuthology.orchestra.run.smithi052.stdout:generic/718 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-05-04T20:32:06.808 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/718 2024-05-04T20:32:06.808 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/718 2024-05-04T20:32:06.808 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:06.817 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:06.827 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:06.827 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/719) 2024-05-04T20:32:07.467 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:07.479 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:07.479 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:07.750 INFO:teuthology.orchestra.run.smithi052.stdout:generic/719 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-05-04T20:32:07.754 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/719 2024-05-04T20:32:07.754 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/719 2024-05-04T20:32:07.754 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:07.756 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:07.776 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:07.777 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/720) 2024-05-04T20:32:08.527 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:08.532 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:08.532 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:08.769 INFO:teuthology.orchestra.run.smithi052.stdout:generic/720 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-05-04T20:32:08.773 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/720 2024-05-04T20:32:08.773 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/720 2024-05-04T20:32:08.773 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:08.776 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:08.794 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:08.794 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/721) 2024-05-04T20:32:09.482 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:09.486 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:09.487 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:09.726 INFO:teuthology.orchestra.run.smithi052.stdout:generic/721 [not run] xfs_io startupdate support is missing 2024-05-04T20:32:09.731 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/721 2024-05-04T20:32:09.731 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/721 2024-05-04T20:32:09.731 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:09.734 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:09.755 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:09.756 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/722) 2024-05-04T20:32:10.059 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:10.066 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:10.066 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:10.345 INFO:teuthology.orchestra.run.smithi052.stdout:generic/722 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-05-04T20:32:10.349 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/722 2024-05-04T20:32:10.349 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/722 2024-05-04T20:32:10.349 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:10.352 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:10.364 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:10.364 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/723) 2024-05-04T20:32:11.062 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:11.068 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:11.069 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:11.340 INFO:teuthology.orchestra.run.smithi052.stdout:generic/723 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-05-04T20:32:11.344 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/723 2024-05-04T20:32:11.344 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/723 2024-05-04T20:32:11.344 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:11.358 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:11.358 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/724) 2024-05-04T20:32:12.093 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:12.099 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:12.099 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:12.342 INFO:teuthology.orchestra.run.smithi052.stdout:generic/724 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-05-04T20:32:12.348 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/724 2024-05-04T20:32:12.348 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/724 2024-05-04T20:32:12.348 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:12.351 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:12.371 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:12.371 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/725) 2024-05-04T20:32:13.022 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:13.027 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:13.300 INFO:teuthology.orchestra.run.smithi052.stdout:generic/725 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-05-04T20:32:13.307 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/725 2024-05-04T20:32:13.307 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/725 2024-05-04T20:32:13.307 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:13.329 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:13.329 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/726) 2024-05-04T20:32:13.989 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:13.995 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:13.995 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:14.665 INFO:teuthology.orchestra.run.smithi052.stdout:generic/726 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-05-04T20:32:14.670 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/726 2024-05-04T20:32:14.670 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/726 2024-05-04T20:32:14.670 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:14.684 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:14.708 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:14.708 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/727) 2024-05-04T20:32:15.332 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:15.338 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:15.339 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:15.692 INFO:teuthology.orchestra.run.smithi052.stdout:generic/727 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-05-04T20:32:15.697 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/727 2024-05-04T20:32:15.697 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/727 2024-05-04T20:32:15.697 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:15.721 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:15.721 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/728) 2024-05-04T20:32:16.437 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:16.444 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:16.444 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:20.796 INFO:teuthology.orchestra.run.smithi052.stdout:generic/728 4s 2024-05-04T20:32:20.804 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/728 2024-05-04T20:32:20.804 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:20.806 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:20.840 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:20.841 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/729) 2024-05-04T20:32:21.107 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:21.123 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:21.123 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:generic/729 0s 2024-05-04T20:32:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/729 2024-05-04T20:32:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:21.840 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:21.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:21.942 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/730) 2024-05-04T20:32:22.573 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:22.579 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:22.579 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:22.806 INFO:teuthology.orchestra.run.smithi052.stdout:generic/730 [not run] require ceph-fuse to be valid block disk 2024-05-04T20:32:22.812 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/730 2024-05-04T20:32:22.812 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/730 2024-05-04T20:32:22.812 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:22.814 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:22.829 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:22.829 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/731) 2024-05-04T20:32:23.128 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:23.133 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:23.133 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:23.313 INFO:teuthology.orchestra.run.smithi052.stdout:generic/731 [not run] require ceph-fuse to be valid block disk 2024-05-04T20:32:23.323 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/731 2024-05-04T20:32:23.323 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/731 2024-05-04T20:32:23.323 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:23.326 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:23.335 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:23.336 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/732) 2024-05-04T20:32:23.698 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:23.703 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:23.704 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:23.907 INFO:teuthology.orchestra.run.smithi052.stdout:generic/732 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:32:23.912 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/732 2024-05-04T20:32:23.912 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/732 2024-05-04T20:32:23.912 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:23.915 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:23.943 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:23.943 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/733) 2024-05-04T20:32:24.242 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:24.258 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:24.258 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:24.445 INFO:teuthology.orchestra.run.smithi052.stdout:generic/733 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:32:24.449 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/733 2024-05-04T20:32:24.449 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/733 2024-05-04T20:32:24.449 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:24.452 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:24.465 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:24.465 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/734) 2024-05-04T20:32:24.757 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:24.772 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:24.772 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:25.006 INFO:teuthology.orchestra.run.smithi052.stdout:generic/734 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-05-04T20:32:25.011 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/734 2024-05-04T20:32:25.011 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/734 2024-05-04T20:32:25.011 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:25.018 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:25.027 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:25.027 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/735) 2024-05-04T20:32:25.647 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:25.652 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:25.652 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:27.324 INFO:teuthology.orchestra.run.smithi052.stdout:generic/735 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-05-04T20:32:27.329 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/735 2024-05-04T20:32:27.329 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/735 2024-05-04T20:32:27.329 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:32:27.338 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:32:27.350 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:32:27.350 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/736) 2024-05-04T20:32:28.029 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:32:28.039 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:32:28.039 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:36:48.414 INFO:teuthology.orchestra.run.smithi052.stdout:generic/736 260s 2024-05-04T20:36:48.466 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/736 2024-05-04T20:36:48.466 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:36:48.469 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:36:48.481 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:36:48.482 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/737) 2024-05-04T20:36:49.194 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:36:49.199 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:36:49.199 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:36:49.474 INFO:teuthology.orchestra.run.smithi052.stdout:generic/737 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:36:49.478 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/737 2024-05-04T20:36:49.478 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/737 2024-05-04T20:36:49.478 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:36:49.481 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:36:49.498 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:36:49.499 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/738) 2024-05-04T20:36:49.831 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:36:49.835 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:36:49.836 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:36:50.011 INFO:teuthology.orchestra.run.smithi052.stdout:generic/738 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:36:50.021 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/738 2024-05-04T20:36:50.021 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/738 2024-05-04T20:36:50.021 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:36:50.035 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:36:50.059 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:36:50.059 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/739) 2024-05-04T20:36:50.483 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:36:50.493 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:36:50.493 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:36:50.743 INFO:teuthology.orchestra.run.smithi052.stdout:generic/739 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:36:50.750 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/739 2024-05-04T20:36:50.750 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/739 2024-05-04T20:36:50.750 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:36:50.753 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:36:50.779 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:36:50.779 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/741) 2024-05-04T20:36:51.077 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:36:51.087 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:36:51.087 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:36:51.670 INFO:teuthology.orchestra.run.smithi052.stdout:generic/741 [not run] this test requires a valid $SCRATCH_DEV 2024-05-04T20:36:51.675 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/741 2024-05-04T20:36:51.675 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/741 2024-05-04T20:36:51.675 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T20:36:51.677 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:36:51.697 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T20:36:51.697 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/742) 2024-05-04T20:36:52.002 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T20:36:52.014 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T20:36:52.014 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T20:41:33.340 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T20:41:33.340+0000 7f7b9124a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 489356) UID: 0 2024-05-04T20:41:33.341 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T20:41:33.340+0000 7f3c99d9c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 489356) UID: 0 2024-05-04T20:41:33.341 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T20:41:33.344+0000 7f17989d7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 489356) UID: 0 2024-05-04T20:41:33.341 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T20:41:33.344+0000 7fd3bf2df640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 489356) UID: 0 2024-05-04T20:45:48.955 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:45:48.962 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:45:49.204 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-05-04T20:45:49.200+0000 7f1412d90640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 37133) UID: 0 2024-05-04T20:45:49.205 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-05-04T20:45:49.204+0000 7fc53dde6640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 37133) UID: 0 2024-05-04T20:45:49.205 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-05-04T20:45:49.200+0000 7fb74db8d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 37133) UID: 0 2024-05-04T20:45:49.207 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-05-04T20:45:49.204+0000 7f81dc32b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 37133) UID: 0 2024-05-04T20:51:39.592 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:51:39.595 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:51:39.609 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-04T20:51:39.604+0000 7f3c99d9c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 489438) UID: 0 2024-05-04T20:51:39.609 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-04T20:51:39.604+0000 7f17989d7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 489438) UID: 0 2024-05-04T20:51:39.609 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-04T20:51:39.604+0000 7fd3bf2df640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 489438) UID: 0 2024-05-04T20:51:39.610 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-04T20:51:39.604+0000 7f7b9124a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 489438) UID: 0 2024-05-04T20:55:51.121 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:55:51.124 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:56:21.294 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:56:21.296 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:56:51.595 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:56:51.597 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:57:21.667 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:57:21.669 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:57:51.741 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:57:51.743 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:58:21.830 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:58:21.832 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:58:51.920 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:58:51.922 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:59:22.096 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:59:22.098 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:59:52.162 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T20:59:52.165 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:00:22.259 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:00:22.262 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:00:52.361 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:00:52.363 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:01:22.768 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:01:22.770 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:01:52.890 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:01:52.892 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:02:23.045 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:02:23.048 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:02:53.092 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:02:53.095 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:03:23.149 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:03:23.152 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:03:53.218 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:03:53.220 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:04:23.376 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:04:23.379 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:04:53.642 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:04:53.645 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:05:23.887 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:05:23.890 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:05:54.004 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:05:54.007 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:06:24.069 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:06:24.071 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:06:54.284 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:06:54.290 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:07:24.375 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:07:24.377 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:07:54.807 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:07:54.811 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:08:24.868 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:08:24.870 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:08:55.008 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:08:55.010 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:09:25.138 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:09:25.141 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:09:55.321 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:09:55.323 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:10:25.425 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:10:25.428 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:10:55.599 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:10:55.601 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:11:25.679 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:11:25.681 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:11:55.819 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:11:55.822 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:12:26.082 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:12:26.085 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:12:56.148 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:12:56.151 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:13:26.203 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:13:26.205 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:13:56.271 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:13:56.274 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:14:26.610 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:14:26.615 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:14:56.856 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:14:56.858 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:15:26.912 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:15:26.915 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:15:56.974 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:15:56.977 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:16:27.036 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:16:27.039 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:16:57.099 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:16:57.102 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:17:27.321 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:17:27.324 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:17:57.371 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:17:57.373 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:18:27.410 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:18:27.413 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:18:57.473 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:18:57.476 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:19:27.528 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:19:27.530 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:19:57.569 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:19:57.576 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:20:13.239 INFO:teuthology.orchestra.run.smithi052.stdout:generic/742 [failed, exit status 1]- output mismatch (see /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/742.out.bad) 2024-05-04T21:20:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: --- tests/generic/742.out 2024-05-04 12:19:32.012992223 +0000 2024-05-04T21:20:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: +++ /tmp/tmp.BtfdD8Y1eWxfstests-dev/results//generic/742.out.bad 2024-05-04 21:20:13.199929936 +0000 2024-05-04T21:20:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: @@ -1,2 +1,3 @@ 2024-05-04T21:20:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: QA output created by 742 2024-05-04T21:20:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: Silence is golden 2024-05-04T21:20:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: +fiemap-fault: fiemap failed 95: Operation not supported 2024-05-04T21:20:13.292 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/742 2024-05-04T21:20:13.293 INFO:teuthology.orchestra.run.smithi052.stdout:Failures: generic/742 2024-05-04T21:20:13.293 INFO:teuthology.orchestra.run.smithi052.stdout:Failed 1 of 1 tests 2024-05-04T21:20:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:13.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T21:20:13.370 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-05-04T21:20:13.371 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-05-04T21:20:13.371 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.BtfdD8Y1eWxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/743) 2024-05-04T21:20:14.219 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph-fuse 2024-05-04T21:20:14.224 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-05-04T21:20:14.224 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:14.474 INFO:teuthology.orchestra.run.smithi052.stdout:generic/743 [not run] xfs_io madvise doesn't support -R 2024-05-04T21:20:14.478 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/743 2024-05-04T21:20:14.478 INFO:teuthology.orchestra.run.smithi052.stdout:Not run: generic/743 2024-05-04T21:20:14.478 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-05-04T21:20:14.480 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:14.491 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-04T21:20:14.491 INFO:tasks.cephfs.xfstests_dev:=========================================================Total number of tests failed = 16========================================================= 2024-05-04T21:20:14.491 DEBUG:teuthology.orchestra.run.smithi052:> sudo userdel --force --remove fsgqa 2024-05-04T21:20:14.692 INFO:teuthology.orchestra.run.smithi052.stderr:userdel: fsgqa mail spool (/var/mail/fsgqa) not found 2024-05-04T21:20:15.096 DEBUG:teuthology.orchestra.run.smithi052:> sudo userdel --force --remove 123456-fsgqa 2024-05-04T21:20:15.126 INFO:teuthology.orchestra.run.smithi052.stderr:userdel: 123456-fsgqa mail spool (/var/mail/123456-fsgqa) not found 2024-05-04T21:20:15.126 INFO:teuthology.orchestra.run.smithi052.stderr:userdel: 123456-fsgqa home directory (/home/123456-fsgqa) not found 2024-05-04T21:20:15.415 DEBUG:teuthology.orchestra.run.smithi052:> sudo groupdel fsgqa 2024-05-04T21:20:15.545 INFO:teuthology.orchestra.run.smithi052.stderr:groupdel: group 'fsgqa' does not exist 2024-05-04T21:20:15.546 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-04T21:20:15.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T21:20:15.547 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo stat /tmp/tmp.BtfdD8Y1eWxfstests-dev/results) 2024-05-04T21:20:15.571 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: cd: /home/ubuntu/cephtest/mnt.0: Permission denied 2024-05-04T21:20:15.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T21:20:15.572 INFO:tasks.cephfs.xfstests_dev:xfstests-dev repo contains not directory named "results". repo location: /tmp/tmp.BtfdD8Y1eWxfstests-dev 2024-05-04T21:20:15.572 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf /tmp/tmp.BtfdD8Y1eWxfstests-dev 2024-05-04T21:20:15.846 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-04T21:20:15.846 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-04T21:20:16.003 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-04T21:20:16.164 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-04T21:20:16.164 INFO:tasks.cephfs.mount:Terminating background process 2024-05-04T21:20:16.165 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T21:20:16.165 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T21:20:16.196 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16818 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-04T21:20:16.196 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T21:20:16.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T21:20:16.197 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-04T21:20:16.251 INFO:tasks.cephfs.mount:Terminating background process 2024-05-04T21:20:16.251 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T21:20:16.251 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T21:20:16.261 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16191 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-04T21:20:16.261 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T21:20:16.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T21:20:16.261 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-04T21:20:16.315 INFO:teuthology.orchestra.run.smithi111.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-04T21:20:16.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T21:20:16.326 INFO:tasks.cephfs.mount:Terminating background process 2024-05-04T21:20:16.326 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T21:20:16.326 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T21:20:16.331 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16818 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-04T21:20:16.331 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T21:20:16.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T21:20:16.331 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-04T21:20:16.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T21:20:16.378 INFO:teuthology.orchestra.run.smithi052.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-04T21:20:16.378 INFO:tasks.cephfs.mount:Terminating background process 2024-05-04T21:20:16.378 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T21:20:16.378 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T21:20:16.384 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16191 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-04T21:20:16.384 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T21:20:16.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T21:20:16.384 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-04T21:20:16.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T21:20:16.431 INFO:teuthology.orchestra.run.smithi111.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-04T21:20:16.431 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T21:20:17.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f72ac640 1 Processor -- start 2024-05-04T21:20:17.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f72ac640 1 -- start start 2024-05-04T21:20:17.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f72ac640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f83f015b7f0 0x7f83f0072800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:17.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f72ac640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83f0070690 0x7f83f0070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:17.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f72ac640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83f00ce740 con 0x7f83f013d9d0 2024-05-04T21:20:17.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f72ac640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f83f00cf030 con 0x7f83f015b7f0 2024-05-04T21:20:17.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f72ac640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f83f00ce9e0 con 0x7f83f0070690 2024-05-04T21:20:17.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f4847640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f83f015b7f0 0x7f83f0072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:17.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f4847640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f83f015b7f0 0x7f83f0072800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36760/0 (socket says 172.21.15.52:36760) 2024-05-04T21:20:17.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f4847640 1 -- 172.21.15.52:0/4090012639 learned_addr learned my addr 172.21.15.52:0/4090012639 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:17.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f5849640 1 --2- 172.21.15.52:0/4090012639 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83f0070690 0x7f83f0070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:17.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f4847640 1 -- 172.21.15.52:0/4090012639 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83f0070690 msgr2=0x7f83f0070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:17.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f4847640 1 --2- 172.21.15.52:0/4090012639 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83f0070690 0x7f83f0070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f4847640 1 -- 172.21.15.52:0/4090012639 >> v1:172.21.15.52:6789/0 conn(0x7f83f013d9d0 legacy=0x7f83f013dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f4847640 1 -- 172.21.15.52:0/4090012639 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f00cecc0 con 0x7f83f015b7f0 2024-05-04T21:20:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.255+0000 7f83f5849640 1 --2- 172.21.15.52:0/4090012639 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83f0070690 0x7f83f0070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T21:20:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.259+0000 7f83f4847640 1 --2- 172.21.15.52:0/4090012639 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f015b7f0 0x7f83f0072800 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f83f00cf1b0 tx=0x7f83e001cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6acdd54a35238f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.263+0000 7f83e7fff640 1 -- 172.21.15.52:0/4090012639 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e00ab650 con 0x7f83f015b7f0 2024-05-04T21:20:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.263+0000 7f83e7fff640 1 -- 172.21.15.52:0/4090012639 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e00b0070 con 0x7f83f015b7f0 2024-05-04T21:20:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.263+0000 7f83e7fff640 1 -- 172.21.15.52:0/4090012639 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e00aba10 con 0x7f83f015b7f0 2024-05-04T21:20:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.263+0000 7f83f72ac640 1 -- 172.21.15.52:0/4090012639 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f015b7f0 msgr2=0x7f83f0072800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.263+0000 7f83f72ac640 1 --2- 172.21.15.52:0/4090012639 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f015b7f0 0x7f83f0072800 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f83f00cf1b0 tx=0x7f83e001cfa0 comp rx=0 tx=0).stop 2024-05-04T21:20:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.263+0000 7f83f72ac640 1 -- 172.21.15.52:0/4090012639 shutdown_connections 2024-05-04T21:20:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.263+0000 7f83f72ac640 1 --2- 172.21.15.52:0/4090012639 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83f0070690 0x7f83f0070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.263+0000 7f83f72ac640 1 --2- 172.21.15.52:0/4090012639 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f015b7f0 0x7f83f0072800 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.263+0000 7f83f72ac640 1 -- 172.21.15.52:0/4090012639 >> 172.21.15.52:0/4090012639 conn(0x7f83f00b2fe0 msgr2=0x7f83f00b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.263+0000 7f83f72ac640 1 -- 172.21.15.52:0/4090012639 shutdown_connections 2024-05-04T21:20:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.263+0000 7f83f72ac640 1 -- 172.21.15.52:0/4090012639 wait complete. 2024-05-04T21:20:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.267+0000 7f83f72ac640 1 Processor -- start 2024-05-04T21:20:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.267+0000 7f83f72ac640 1 -- start start 2024-05-04T21:20:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.267+0000 7f83f72ac640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83f0070690 0x7f83f0156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.267+0000 7f83f72ac640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f015b7f0 0x7f83f013f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.267+0000 7f83f72ac640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83f013c4f0 0x7f83f013c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.267+0000 7f83f72ac640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83f00cfe30 con 0x7f83f015b7f0 2024-05-04T21:20:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.267+0000 7f83f72ac640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f83f00cfcb0 con 0x7f83f013c4f0 2024-05-04T21:20:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.267+0000 7f83f72ac640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f83f00cffb0 con 0x7f83f0070690 2024-05-04T21:20:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83f4847640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f015b7f0 0x7f83f013f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83f4847640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f015b7f0 0x7f83f013f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36770/0 (socket says 172.21.15.52:36770) 2024-05-04T21:20:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83f4847640 1 -- 172.21.15.52:0/3450012286 learned_addr learned my addr 172.21.15.52:0/3450012286 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83f5849640 1 --2- 172.21.15.52:0/3450012286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83f013c4f0 0x7f83f013c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83f4847640 1 -- 172.21.15.52:0/3450012286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83f0070690 msgr2=0x7f83f0156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83f4847640 1 --2- 172.21.15.52:0/3450012286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83f0070690 0x7f83f0156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83f4847640 1 -- 172.21.15.52:0/3450012286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83f013c4f0 msgr2=0x7f83f013c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83f4847640 1 --2- 172.21.15.52:0/3450012286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83f013c4f0 0x7f83f013c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83f4847640 1 -- 172.21.15.52:0/3450012286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f0156aa0 con 0x7f83f015b7f0 2024-05-04T21:20:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83f4847640 1 --2- 172.21.15.52:0/3450012286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f015b7f0 0x7f83f013f920 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f83e00ad040 tx=0x7f83e00192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83e5ffb640 1 -- 172.21.15.52:0/3450012286 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e00bf610 con 0x7f83f015b7f0 2024-05-04T21:20:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83e5ffb640 1 -- 172.21.15.52:0/3450012286 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e00bf7d0 con 0x7f83f015b7f0 2024-05-04T21:20:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83f72ac640 1 -- 172.21.15.52:0/3450012286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f013d550 con 0x7f83f015b7f0 2024-05-04T21:20:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83e5ffb640 1 -- 172.21.15.52:0/3450012286 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e00bfbc0 con 0x7f83f015b7f0 2024-05-04T21:20:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83f72ac640 1 -- 172.21.15.52:0/3450012286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f013d270 con 0x7f83f015b7f0 2024-05-04T21:20:17.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.271+0000 7f83db7fe640 1 -- 172.21.15.52:0/3450012286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c80052a0 con 0x7f83f015b7f0 2024-05-04T21:20:17.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.347+0000 7f83e5ffb640 1 -- 172.21.15.52:0/3450012286 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f83e00bd020 con 0x7f83f015b7f0 2024-05-04T21:20:17.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.347+0000 7f83e5ffb640 1 --2- 172.21.15.52:0/3450012286 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f83d40678c0 0x7f83d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:17.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.347+0000 7f83e5ffb640 1 -- 172.21.15.52:0/3450012286 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f83e01206f0 con 0x7f83f015b7f0 2024-05-04T21:20:17.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.347+0000 7f83f5048640 1 --2- 172.21.15.52:0/3450012286 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f83d40678c0 0x7f83d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:17.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.347+0000 7f83f5048640 1 --2- 172.21.15.52:0/3450012286 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f83d40678c0 0x7f83d4069d80 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f83ec000f70 tx=0x7f83ec0079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:17.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.611+0000 7f83e5ffb640 1 -- 172.21.15.52:0/3450012286 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f83e0016130 con 0x7f83f015b7f0 2024-05-04T21:20:17.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.895+0000 7f83db7fe640 1 -- 172.21.15.52:0/3450012286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83c8005590 con 0x7f83f015b7f0 2024-05-04T21:20:17.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.915+0000 7f83e5ffb640 1 -- 172.21.15.52:0/3450012286 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5184 (secure 0 0 0) 0x7f83e00f0d50 con 0x7f83f015b7f0 2024-05-04T21:20:17.922 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:17.922 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2438654614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2438654614},{"type":"v1","addr":"172.21.15.111:6835","nonce":2438654614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/796226117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":796226117},{"type":"v1","addr":"172.21.15.52:6835","nonce":796226117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3866938727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3866938727},{"type":"v1","addr":"172.21.15.111:6833","nonce":3866938727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-04T12:19:22.017880+0000","modified":"2024-05-04T12:19:25.052316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5199":{"gid":5199,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6837/1620779861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1620779861},{"type":"v1","addr":"172.21.15.52:6837","nonce":1620779861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5199,"qdb_cluster":[5199]},"id":2}]} 2024-05-04T21:20:17.922 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 16 2024-05-04T21:20:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.919+0000 7f83db7fe640 1 -- 172.21.15.52:0/3450012286 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f83d40678c0 msgr2=0x7f83d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.919+0000 7f83db7fe640 1 --2- 172.21.15.52:0/3450012286 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f83d40678c0 0x7f83d4069d80 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f83ec000f70 tx=0x7f83ec0079e0 comp rx=0 tx=0).stop 2024-05-04T21:20:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.919+0000 7f83db7fe640 1 -- 172.21.15.52:0/3450012286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f015b7f0 msgr2=0x7f83f013f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.919+0000 7f83db7fe640 1 --2- 172.21.15.52:0/3450012286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f015b7f0 0x7f83f013f920 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f83e00ad040 tx=0x7f83e00192e0 comp rx=0 tx=0).stop 2024-05-04T21:20:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.919+0000 7f83db7fe640 1 -- 172.21.15.52:0/3450012286 shutdown_connections 2024-05-04T21:20:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.919+0000 7f83db7fe640 1 --2- 172.21.15.52:0/3450012286 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f83d40678c0 0x7f83d4069d80 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.919+0000 7f83db7fe640 1 --2- 172.21.15.52:0/3450012286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83f013c4f0 0x7f83f013c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.919+0000 7f83db7fe640 1 --2- 172.21.15.52:0/3450012286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f015b7f0 0x7f83f013f920 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.919+0000 7f83db7fe640 1 --2- 172.21.15.52:0/3450012286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83f0070690 0x7f83f0156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.919+0000 7f83db7fe640 1 -- 172.21.15.52:0/3450012286 >> 172.21.15.52:0/3450012286 conn(0x7f83f00b2fe0 msgr2=0x7f83f015c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.919+0000 7f83db7fe640 1 -- 172.21.15.52:0/3450012286 shutdown_connections 2024-05-04T21:20:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:17.919+0000 7f83db7fe640 1 -- 172.21.15.52:0/3450012286 wait complete. 2024-05-04T21:20:17.963 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T21:20:18.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.099+0000 7fd81c0ab640 1 Processor -- start 2024-05-04T21:20:18.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.099+0000 7fd81c0ab640 1 -- start start 2024-05-04T21:20:18.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.099+0000 7fd81c0ab640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd81413dec0 0x7fd814072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:18.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.099+0000 7fd81c0ab640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd8140733c0 0x7fd81406def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:18.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.099+0000 7fd81c0ab640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd81413e630 con 0x7fd81406e430 2024-05-04T21:20:18.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.099+0000 7fd81c0ab640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fd81406ec90 con 0x7fd8140733c0 2024-05-04T21:20:18.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.099+0000 7fd81c0ab640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fd81406ef70 con 0x7fd81413dec0 2024-05-04T21:20:18.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.099+0000 7fd819e47640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd81413dec0 0x7fd814072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:18.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.099+0000 7fd819e47640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd81413dec0 0x7fd814072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:47720/0 (socket says 172.21.15.52:47720) 2024-05-04T21:20:18.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.099+0000 7fd819e47640 1 -- 172.21.15.52:0/280672522 learned_addr learned my addr 172.21.15.52:0/280672522 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:18.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.099+0000 7fd819e47640 1 -- 172.21.15.52:0/280672522 >> v2:172.21.15.52:3300/0 conn(0x7fd8140733c0 msgr2=0x7fd81406def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:18.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.099+0000 7fd819e47640 1 --2- 172.21.15.52:0/280672522 >> v2:172.21.15.52:3300/0 conn(0x7fd8140733c0 0x7fd81406def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:18.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.099+0000 7fd819e47640 1 -- 172.21.15.52:0/280672522 >> v1:172.21.15.111:6789/0 conn(0x7fd81406e430 legacy=0x7fd814140660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:18.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.099+0000 7fd819e47640 1 -- 172.21.15.52:0/280672522 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81406f250 con 0x7fd81413dec0 2024-05-04T21:20:18.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.103+0000 7fd819e47640 1 --2- 172.21.15.52:0/280672522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81413dec0 0x7fd814072e80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd8140719e0 tx=0x7fd81001eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d082b1bea6bf42ef server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:18.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.103+0000 7fd818e45640 1 -- 172.21.15.52:0/280672522 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8100b4070 con 0x7fd81413dec0 2024-05-04T21:20:18.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.103+0000 7fd818e45640 1 -- 172.21.15.52:0/280672522 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8100a7a80 con 0x7fd81413dec0 2024-05-04T21:20:18.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.103+0000 7fd818e45640 1 -- 172.21.15.52:0/280672522 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8100a7e50 con 0x7fd81413dec0 2024-05-04T21:20:18.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.103+0000 7fd81c0ab640 1 -- 172.21.15.52:0/280672522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81413dec0 msgr2=0x7fd814072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:18.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.103+0000 7fd81c0ab640 1 --2- 172.21.15.52:0/280672522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81413dec0 0x7fd814072e80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd8140719e0 tx=0x7fd81001eb20 comp rx=0 tx=0).stop 2024-05-04T21:20:18.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.103+0000 7fd81c0ab640 1 -- 172.21.15.52:0/280672522 shutdown_connections 2024-05-04T21:20:18.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.103+0000 7fd81c0ab640 1 --2- 172.21.15.52:0/280672522 >> v2:172.21.15.52:3300/0 conn(0x7fd8140733c0 0x7fd81406def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:18.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.103+0000 7fd81c0ab640 1 --2- 172.21.15.52:0/280672522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81413dec0 0x7fd814072e80 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:18.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.103+0000 7fd81c0ab640 1 -- 172.21.15.52:0/280672522 >> 172.21.15.52:0/280672522 conn(0x7fd814067070 msgr2=0x7fd814067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:18.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.103+0000 7fd81c0ab640 1 -- 172.21.15.52:0/280672522 shutdown_connections 2024-05-04T21:20:18.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.103+0000 7fd81c0ab640 1 -- 172.21.15.52:0/280672522 wait complete. 2024-05-04T21:20:18.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd81c0ab640 1 Processor -- start 2024-05-04T21:20:18.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd81c0ab640 1 -- start start 2024-05-04T21:20:18.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd81c0ab640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd81406e430 0x7fd8140d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:18.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd81c0ab640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8140733c0 0x7fd814154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:18.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd81c0ab640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81413dec0 0x7fd8141574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:18.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd81c0ab640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd814070850 con 0x7fd8140733c0 2024-05-04T21:20:18.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd81c0ab640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd8140706d0 con 0x7fd81413dec0 2024-05-04T21:20:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd81c0ab640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fd8140709d0 con 0x7fd81406e430 2024-05-04T21:20:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd819646640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8140733c0 0x7fd814154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd819e47640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd81406e430 0x7fd8140d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd819646640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8140733c0 0x7fd814154dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36782/0 (socket says 172.21.15.52:36782) 2024-05-04T21:20:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd819646640 1 -- 172.21.15.52:0/2938919751 learned_addr learned my addr 172.21.15.52:0/2938919751 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd81a648640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81413dec0 0x7fd8141574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd819e47640 1 -- 172.21.15.52:0/2938919751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81413dec0 msgr2=0x7fd8141574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd819e47640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81413dec0 0x7fd8141574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd819e47640 1 -- 172.21.15.52:0/2938919751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8140733c0 msgr2=0x7fd814154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd819e47640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8140733c0 0x7fd814154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd819e47640 1 -- 172.21.15.52:0/2938919751 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd814155300 con 0x7fd81406e430 2024-05-04T21:20:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd81a648640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81413dec0 0x7fd8141574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T21:20:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd819646640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8140733c0 0x7fd814154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T21:20:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd819e47640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd81406e430 0x7fd8140d04b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd8100ae000 tx=0x7fd81001d360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:18.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd80a7fc640 1 -- 172.21.15.52:0/2938919751 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8100bf070 con 0x7fd81406e430 2024-05-04T21:20:18.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd80a7fc640 1 -- 172.21.15.52:0/2938919751 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8100ba640 con 0x7fd81406e430 2024-05-04T21:20:18.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd81c0ab640 1 -- 172.21.15.52:0/2938919751 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8140d0c20 con 0x7fd81406e430 2024-05-04T21:20:18.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd80a7fc640 1 -- 172.21.15.52:0/2938919751 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8100b4070 con 0x7fd81406e430 2024-05-04T21:20:18.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.107+0000 7fd81c0ab640 1 -- 172.21.15.52:0/2938919751 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8141542e0 con 0x7fd81406e430 2024-05-04T21:20:18.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.111+0000 7fd81c0ab640 1 -- 172.21.15.52:0/2938919751 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd814062e50 con 0x7fd81406e430 2024-05-04T21:20:18.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.135+0000 7fd80a7fc640 1 -- 172.21.15.52:0/2938919751 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd8100c2070 con 0x7fd81406e430 2024-05-04T21:20:18.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.135+0000 7fd80a7fc640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd7f80678c0 0x7fd7f8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:18.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.135+0000 7fd80a7fc640 1 -- 172.21.15.52:0/2938919751 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fd810120530 con 0x7fd81406e430 2024-05-04T21:20:18.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.135+0000 7fd819646640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd7f80678c0 0x7fd7f8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:18.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.135+0000 7fd819646640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd7f80678c0 0x7fd7f8069d80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd8141faac0 tx=0x7fd804009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:18.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.295+0000 7fd80a7fc640 1 -- 172.21.15.52:0/2938919751 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd814155300 con 0x7fd81406e430 2024-05-04T21:20:18.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.539+0000 7fd81c0ab640 1 -- 172.21.15.52:0/2938919751 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd814155530 con 0x7fd81406e430 2024-05-04T21:20:18.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.639+0000 7fd80a7fc640 1 -- 172.21.15.52:0/2938919751 <== mon.2 v2:172.21.15.111:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16681 (secure 0 0 0) 0x7fd8100187a0 con 0x7fd81406e430 2024-05-04T21:20:18.646 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:18.646 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":34,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:19:23.639019+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:19:20.589701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-04T12:19:20.997186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T21:20:18.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.643+0000 7fd81c0ab640 1 -- 172.21.15.52:0/2938919751 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd7f80678c0 msgr2=0x7fd7f8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:18.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.643+0000 7fd81c0ab640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd7f80678c0 0x7fd7f8069d80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd8141faac0 tx=0x7fd804009210 comp rx=0 tx=0).stop 2024-05-04T21:20:18.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.643+0000 7fd81c0ab640 1 -- 172.21.15.52:0/2938919751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd81406e430 msgr2=0x7fd8140d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:18.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.643+0000 7fd81c0ab640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd81406e430 0x7fd8140d04b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd8100ae000 tx=0x7fd81001d360 comp rx=0 tx=0).stop 2024-05-04T21:20:18.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.643+0000 7fd81c0ab640 1 -- 172.21.15.52:0/2938919751 shutdown_connections 2024-05-04T21:20:18.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.643+0000 7fd81c0ab640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd7f80678c0 0x7fd7f8069d80 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:18.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.643+0000 7fd81c0ab640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81413dec0 0x7fd8141574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:18.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.643+0000 7fd81c0ab640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8140733c0 0x7fd814154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:18.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.643+0000 7fd81c0ab640 1 --2- 172.21.15.52:0/2938919751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd81406e430 0x7fd8140d04b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:18.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.643+0000 7fd81c0ab640 1 -- 172.21.15.52:0/2938919751 >> 172.21.15.52:0/2938919751 conn(0x7fd814067070 msgr2=0x7fd81415a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:18.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.643+0000 7fd81c0ab640 1 -- 172.21.15.52:0/2938919751 shutdown_connections 2024-05-04T21:20:18.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.643+0000 7fd81c0ab640 1 -- 172.21.15.52:0/2938919751 wait complete. 2024-05-04T21:20:18.672 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T12:18:10.817553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T12:19:20.589701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T12:19:20.997186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T21:20:18.672 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T21:20:18.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a6b16b640 1 Processor -- start 2024-05-04T21:20:18.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a6b16b640 1 -- start start 2024-05-04T21:20:18.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a6b16b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0a640b2df0 0x7f0a6415b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:18.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a6b16b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a640c4280 0x7f0a6406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:18.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a6b16b640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a6415bad0 con 0x7f0a6415e970 2024-05-04T21:20:18.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a6b16b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f0a640cd950 con 0x7f0a640b2df0 2024-05-04T21:20:18.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a6b16b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f0a6415c370 con 0x7f0a640c4280 2024-05-04T21:20:18.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a63fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f0a6415e970 0x7f0a640c3b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:45078/0 (socket says 172.21.15.52:45078) 2024-05-04T21:20:18.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a63fff640 1 -- 172.21.15.52:0/297000343 learned_addr learned my addr 172.21.15.52:0/297000343 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:18.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a69708640 1 --2- 172.21.15.52:0/297000343 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a640c4280 0x7f0a6406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:18.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a68f07640 1 --2- 172.21.15.52:0/297000343 >> v2:172.21.15.111:3300/0 conn(0x7f0a640b2df0 0x7f0a6415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:18.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a637fe640 1 -- 172.21.15.52:0/297000343 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 704955906 0 0) 0x7f0a6415bad0 con 0x7f0a6415e970 2024-05-04T21:20:18.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a637fe640 1 -- 172.21.15.52:0/297000343 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a54003530 con 0x7f0a6415e970 2024-05-04T21:20:18.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a68f07640 1 -- 172.21.15.52:0/297000343 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a640c4280 msgr2=0x7f0a6406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:18.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a68f07640 1 --2- 172.21.15.52:0/297000343 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a640c4280 0x7f0a6406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:18.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a68f07640 1 -- 172.21.15.52:0/297000343 >> v1:172.21.15.52:6789/0 conn(0x7f0a6415e970 legacy=0x7f0a640c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:18.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.815+0000 7f0a68f07640 1 -- 172.21.15.52:0/297000343 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6415bdb0 con 0x7f0a640b2df0 2024-05-04T21:20:18.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.819+0000 7f0a68f07640 1 --2- 172.21.15.52:0/297000343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a640b2df0 0x7f0a6415b380 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0a580097c0 tx=0x7f0a5801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c768e6f4555280 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:18.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.819+0000 7f0a637fe640 1 -- 172.21.15.52:0/297000343 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a580b0070 con 0x7f0a640b2df0 2024-05-04T21:20:18.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.819+0000 7f0a637fe640 1 -- 172.21.15.52:0/297000343 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a580ab600 con 0x7f0a640b2df0 2024-05-04T21:20:18.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.819+0000 7f0a637fe640 1 -- 172.21.15.52:0/297000343 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a580ab9d0 con 0x7f0a640b2df0 2024-05-04T21:20:18.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.819+0000 7f0a6b16b640 1 -- 172.21.15.52:0/297000343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a640b2df0 msgr2=0x7f0a6415b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:18.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.819+0000 7f0a6b16b640 1 --2- 172.21.15.52:0/297000343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a640b2df0 0x7f0a6415b380 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0a580097c0 tx=0x7f0a5801c410 comp rx=0 tx=0).stop 2024-05-04T21:20:18.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.823+0000 7f0a6b16b640 1 -- 172.21.15.52:0/297000343 shutdown_connections 2024-05-04T21:20:18.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.823+0000 7f0a6b16b640 1 --2- 172.21.15.52:0/297000343 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a640c4280 0x7f0a6406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:18.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.823+0000 7f0a6b16b640 1 --2- 172.21.15.52:0/297000343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a640b2df0 0x7f0a6415b380 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:18.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.823+0000 7f0a6b16b640 1 -- 172.21.15.52:0/297000343 >> 172.21.15.52:0/297000343 conn(0x7f0a640d12c0 msgr2=0x7f0a640d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:18.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a6b16b640 1 -- 172.21.15.52:0/297000343 shutdown_connections 2024-05-04T21:20:18.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a6b16b640 1 -- 172.21.15.52:0/297000343 wait complete. 2024-05-04T21:20:18.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a6b16b640 1 Processor -- start 2024-05-04T21:20:18.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a6b16b640 1 -- start start 2024-05-04T21:20:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a6b16b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a640b2df0 0x7f0a6413b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a6b16b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a640c4280 0x7f0a64072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a6b16b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a6415e970 0x7f0a64140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a6b16b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6406f280 con 0x7f0a640b2df0 2024-05-04T21:20:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a6b16b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6406f100 con 0x7f0a6415e970 2024-05-04T21:20:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a6b16b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f0a6406f400 con 0x7f0a640c4280 2024-05-04T21:20:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a68f07640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a640b2df0 0x7f0a6413b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a68f07640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a640b2df0 0x7f0a6413b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36792/0 (socket says 172.21.15.52:36792) 2024-05-04T21:20:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a68f07640 1 -- 172.21.15.52:0/2858125614 learned_addr learned my addr 172.21.15.52:0/2858125614 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a63fff640 1 --2- 172.21.15.52:0/2858125614 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a640c4280 0x7f0a64072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a69708640 1 --2- 172.21.15.52:0/2858125614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a6415e970 0x7f0a64140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a68f07640 1 -- 172.21.15.52:0/2858125614 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a640c4280 msgr2=0x7f0a64072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a68f07640 1 --2- 172.21.15.52:0/2858125614 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a640c4280 0x7f0a64072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a68f07640 1 -- 172.21.15.52:0/2858125614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a6415e970 msgr2=0x7f0a64140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a68f07640 1 --2- 172.21.15.52:0/2858125614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a6415e970 0x7f0a64140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a68f07640 1 -- 172.21.15.52:0/2858125614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6413a030 con 0x7f0a640b2df0 2024-05-04T21:20:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.827+0000 7f0a68f07640 1 --2- 172.21.15.52:0/2858125614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a640b2df0 0x7f0a6413b0d0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f0a580ae040 tx=0x7f0a5801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.831+0000 7f0a617fa640 1 -- 172.21.15.52:0/2858125614 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a580b0050 con 0x7f0a640b2df0 2024-05-04T21:20:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.831+0000 7f0a617fa640 1 -- 172.21.15.52:0/2858125614 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a580bc070 con 0x7f0a640b2df0 2024-05-04T21:20:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.831+0000 7f0a6b16b640 1 -- 172.21.15.52:0/2858125614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6413a310 con 0x7f0a640b2df0 2024-05-04T21:20:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.831+0000 7f0a617fa640 1 -- 172.21.15.52:0/2858125614 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58006e20 con 0x7f0a640b2df0 2024-05-04T21:20:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.831+0000 7f0a6b16b640 1 -- 172.21.15.52:0/2858125614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a6413a8d0 con 0x7f0a640b2df0 2024-05-04T21:20:18.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.831+0000 7f0a6b16b640 1 -- 172.21.15.52:0/2858125614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a64062e50 con 0x7f0a640b2df0 2024-05-04T21:20:18.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.831+0000 7f0a617fa640 1 -- 172.21.15.52:0/2858125614 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0a580b5870 con 0x7f0a640b2df0 2024-05-04T21:20:18.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.831+0000 7f0a617fa640 1 --2- 172.21.15.52:0/2858125614 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0a480678f0 0x7f0a48069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:18.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.831+0000 7f0a617fa640 1 -- 172.21.15.52:0/2858125614 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0a58120260 con 0x7f0a640b2df0 2024-05-04T21:20:18.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.831+0000 7f0a63fff640 1 --2- 172.21.15.52:0/2858125614 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0a480678f0 0x7f0a48069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:18.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.831+0000 7f0a63fff640 1 --2- 172.21.15.52:0/2858125614 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0a480678f0 0x7f0a48069db0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0a50007fd0 tx=0x7f0a50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:18.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:18.875+0000 7f0a617fa640 1 -- 172.21.15.52:0/2858125614 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0a58016090 con 0x7f0a640b2df0 2024-05-04T21:20:19.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.107+0000 7f0a6b16b640 1 -- 172.21.15.52:0/2858125614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0a64075320 con 0x7f0a640b2df0 2024-05-04T21:20:19.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.163+0000 7f0a617fa640 1 -- 172.21.15.52:0/2858125614 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f0a580f0750 con 0x7f0a640b2df0 2024-05-04T21:20:19.171 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-04T21:20:19.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.167+0000 7f0a6b16b640 1 -- 172.21.15.52:0/2858125614 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0a480678f0 msgr2=0x7f0a48069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:19.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.167+0000 7f0a6b16b640 1 --2- 172.21.15.52:0/2858125614 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0a480678f0 0x7f0a48069db0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0a50007fd0 tx=0x7f0a50009040 comp rx=0 tx=0).stop 2024-05-04T21:20:19.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.167+0000 7f0a6b16b640 1 -- 172.21.15.52:0/2858125614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a640b2df0 msgr2=0x7f0a6413b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:19.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.167+0000 7f0a6b16b640 1 --2- 172.21.15.52:0/2858125614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a640b2df0 0x7f0a6413b0d0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f0a580ae040 tx=0x7f0a5801ac50 comp rx=0 tx=0).stop 2024-05-04T21:20:19.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.171+0000 7f0a6b16b640 1 -- 172.21.15.52:0/2858125614 shutdown_connections 2024-05-04T21:20:19.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.171+0000 7f0a6b16b640 1 --2- 172.21.15.52:0/2858125614 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f0a480678f0 0x7f0a48069db0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.171+0000 7f0a6b16b640 1 --2- 172.21.15.52:0/2858125614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a6415e970 0x7f0a64140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.171+0000 7f0a6b16b640 1 --2- 172.21.15.52:0/2858125614 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a640c4280 0x7f0a64072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.171+0000 7f0a6b16b640 1 --2- 172.21.15.52:0/2858125614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a640b2df0 0x7f0a6413b0d0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.171+0000 7f0a6b16b640 1 -- 172.21.15.52:0/2858125614 >> 172.21.15.52:0/2858125614 conn(0x7f0a640d12c0 msgr2=0x7f0a6415cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:19.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.171+0000 7f0a6b16b640 1 -- 172.21.15.52:0/2858125614 shutdown_connections 2024-05-04T21:20:19.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.171+0000 7f0a6b16b640 1 -- 172.21.15.52:0/2858125614 wait complete. 2024-05-04T21:20:19.206 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T21:20:19.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6efe605640 1 Processor -- start 2024-05-04T21:20:19.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6efe605640 1 -- start start 2024-05-04T21:20:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6efe605640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6ef815a3a0 0x7f6ef80cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6efe605640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ef80cc450 0x7f6ef80ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6efe605640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef806d290 con 0x7f6ef80cd020 2024-05-04T21:20:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6efe605640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f6ef806e730 con 0x7f6ef815a3a0 2024-05-04T21:20:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6efe605640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef815e990 con 0x7f6ef80cc450 2024-05-04T21:20:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6ef77fe640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ef80cc450 0x7f6ef80ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6ef7fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6ef815a3a0 0x7f6ef80cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6ef7fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6ef815a3a0 0x7f6ef80cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:47748/0 (socket says 172.21.15.52:47748) 2024-05-04T21:20:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6ef77fe640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ef80cc450 0x7f6ef80ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.52:35264/0 (socket says 172.21.15.52:35264) 2024-05-04T21:20:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6ef7fff640 1 -- 172.21.15.52:0/800472171 learned_addr learned my addr 172.21.15.52:0/800472171 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:19.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6ef7fff640 1 -- 172.21.15.52:0/800472171 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ef80cc450 msgr2=0x7f6ef80ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:19.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6ef7fff640 1 --2- 172.21.15.52:0/800472171 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ef80cc450 0x7f6ef80ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6ef7fff640 1 -- 172.21.15.52:0/800472171 >> v1:172.21.15.52:6789/0 conn(0x7f6ef80cd020 legacy=0x7f6ef80cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:19.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6ef7fff640 1 -- 172.21.15.52:0/800472171 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef815ec70 con 0x7f6ef815a3a0 2024-05-04T21:20:19.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.363+0000 7f6ef77fe640 1 --2- 172.21.15.52:0/800472171 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ef80cc450 0x7f6ef80ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T21:20:19.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.367+0000 7f6ef7fff640 1 --2- 172.21.15.52:0/800472171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ef815a3a0 0x7f6ef80cbe00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000bef0 tx=0x7f6ef001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=536446dceb40ef05 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:19.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.367+0000 7f6ef6ffd640 1 -- 172.21.15.52:0/800472171 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00ab070 con 0x7f6ef815a3a0 2024-05-04T21:20:19.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.367+0000 7f6ef6ffd640 1 -- 172.21.15.52:0/800472171 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef00a58d0 con 0x7f6ef815a3a0 2024-05-04T21:20:19.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.367+0000 7f6ef6ffd640 1 -- 172.21.15.52:0/800472171 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00a5cc0 con 0x7f6ef815a3a0 2024-05-04T21:20:19.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.367+0000 7f6efe605640 1 -- 172.21.15.52:0/800472171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ef815a3a0 msgr2=0x7f6ef80cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:19.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.367+0000 7f6efe605640 1 --2- 172.21.15.52:0/800472171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ef815a3a0 0x7f6ef80cbe00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000bef0 tx=0x7f6ef001c940 comp rx=0 tx=0).stop 2024-05-04T21:20:19.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.367+0000 7f6efe605640 1 -- 172.21.15.52:0/800472171 shutdown_connections 2024-05-04T21:20:19.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.367+0000 7f6efe605640 1 --2- 172.21.15.52:0/800472171 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ef80cc450 0x7f6ef80ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.367+0000 7f6efe605640 1 --2- 172.21.15.52:0/800472171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ef815a3a0 0x7f6ef80cbe00 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.367+0000 7f6efe605640 1 -- 172.21.15.52:0/800472171 >> 172.21.15.52:0/800472171 conn(0x7f6ef80672a0 msgr2=0x7f6ef80b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:19.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.371+0000 7f6efe605640 1 -- 172.21.15.52:0/800472171 shutdown_connections 2024-05-04T21:20:19.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.371+0000 7f6efe605640 1 -- 172.21.15.52:0/800472171 wait complete. 2024-05-04T21:20:19.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.371+0000 7f6efe605640 1 Processor -- start 2024-05-04T21:20:19.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.371+0000 7f6efe605640 1 -- start start 2024-05-04T21:20:19.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.371+0000 7f6efe605640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ef80cc450 0x7f6ef814a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:19.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.371+0000 7f6efe605640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ef80cd020 0x7f6ef815a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:19.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.371+0000 7f6efe605640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ef815a3a0 0x7f6ef80ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:19.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.371+0000 7f6efe605640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef81671f0 con 0x7f6ef815a3a0 2024-05-04T21:20:19.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.371+0000 7f6efe605640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8167370 con 0x7f6ef80cc450 2024-05-04T21:20:19.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.371+0000 7f6efe605640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef81674f0 con 0x7f6ef80cd020 2024-05-04T21:20:19.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.371+0000 7f6ef7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ef80cc450 0x7f6ef814a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:19.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.371+0000 7f6efcba2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ef815a3a0 0x7f6ef80ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:19.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.371+0000 7f6efcba2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ef815a3a0 0x7f6ef80ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36808/0 (socket says 172.21.15.52:36808) 2024-05-04T21:20:19.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.371+0000 7f6efcba2640 1 -- 172.21.15.52:0/258899580 learned_addr learned my addr 172.21.15.52:0/258899580 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:19.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.375+0000 7f6ef7fff640 1 -- 172.21.15.52:0/258899580 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ef80cd020 msgr2=0x7f6ef815a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:19.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.375+0000 7f6ef7fff640 1 --2- 172.21.15.52:0/258899580 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ef80cd020 0x7f6ef815a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.375+0000 7f6ef7fff640 1 -- 172.21.15.52:0/258899580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ef815a3a0 msgr2=0x7f6ef80ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:19.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.375+0000 7f6ef7fff640 1 --2- 172.21.15.52:0/258899580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ef815a3a0 0x7f6ef80ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.375+0000 7f6ef7fff640 1 -- 172.21.15.52:0/258899580 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef815b320 con 0x7f6ef80cc450 2024-05-04T21:20:19.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.375+0000 7f6efcba2640 1 --2- 172.21.15.52:0/258899580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ef815a3a0 0x7f6ef80ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T21:20:19.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.375+0000 7f6ef7fff640 1 --2- 172.21.15.52:0/258899580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ef80cc450 0x7f6ef814a880 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00ad000 tx=0x7f6ef000a4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:19.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.375+0000 7f6ef4ff9640 1 -- 172.21.15.52:0/258899580 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00bd020 con 0x7f6ef80cc450 2024-05-04T21:20:19.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.375+0000 7f6ef4ff9640 1 -- 172.21.15.52:0/258899580 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef00039a0 con 0x7f6ef80cc450 2024-05-04T21:20:19.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.375+0000 7f6efe605640 1 -- 172.21.15.52:0/258899580 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef815b600 con 0x7f6ef80cc450 2024-05-04T21:20:19.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.375+0000 7f6ef4ff9640 1 -- 172.21.15.52:0/258899580 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00ab040 con 0x7f6ef80cc450 2024-05-04T21:20:19.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.375+0000 7f6efe605640 1 -- 172.21.15.52:0/258899580 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef815bbc0 con 0x7f6ef80cc450 2024-05-04T21:20:19.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.375+0000 7f6efe605640 1 -- 172.21.15.52:0/258899580 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef8062e50 con 0x7f6ef80cc450 2024-05-04T21:20:19.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.379+0000 7f6ef4ff9640 1 -- 172.21.15.52:0/258899580 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6ef00c0020 con 0x7f6ef80cc450 2024-05-04T21:20:19.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.379+0000 7f6ef4ff9640 1 --2- 172.21.15.52:0/258899580 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6edc0678c0 0x7f6edc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:19.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.379+0000 7f6ef4ff9640 1 -- 172.21.15.52:0/258899580 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f6ef0120550 con 0x7f6ef80cc450 2024-05-04T21:20:19.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.379+0000 7f6ef77fe640 1 --2- 172.21.15.52:0/258899580 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6edc0678c0 0x7f6edc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:19.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.379+0000 7f6ef77fe640 1 --2- 172.21.15.52:0/258899580 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6edc0678c0 0x7f6edc069d80 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8152f30 tx=0x7f6eec00f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:19.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.451+0000 7f6ef4ff9640 1 -- 172.21.15.52:0/258899580 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6ef00c7070 con 0x7f6ef80cc450 2024-05-04T21:20:19.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.691+0000 7f6efe605640 1 -- 172.21.15.52:0/258899580 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6ef8075320 con 0x7f6ef80cc450 2024-05-04T21:20:19.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.731+0000 7f6ef4ff9640 1 -- 172.21.15.52:0/258899580 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f6ef0016650 con 0x7f6ef80cc450 2024-05-04T21:20:19.735 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-04T21:20:19.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.731+0000 7f6efe605640 1 -- 172.21.15.52:0/258899580 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6edc0678c0 msgr2=0x7f6edc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.731+0000 7f6efe605640 1 --2- 172.21.15.52:0/258899580 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6edc0678c0 0x7f6edc069d80 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8152f30 tx=0x7f6eec00f040 comp rx=0 tx=0).stop 2024-05-04T21:20:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.731+0000 7f6efe605640 1 -- 172.21.15.52:0/258899580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ef80cc450 msgr2=0x7f6ef814a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.731+0000 7f6efe605640 1 --2- 172.21.15.52:0/258899580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ef80cc450 0x7f6ef814a880 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00ad000 tx=0x7f6ef000a4c0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.731+0000 7f6efe605640 1 -- 172.21.15.52:0/258899580 shutdown_connections 2024-05-04T21:20:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.731+0000 7f6efe605640 1 --2- 172.21.15.52:0/258899580 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f6edc0678c0 0x7f6edc069d80 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.731+0000 7f6efe605640 1 --2- 172.21.15.52:0/258899580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ef815a3a0 0x7f6ef80ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.731+0000 7f6efe605640 1 --2- 172.21.15.52:0/258899580 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ef80cd020 0x7f6ef815a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.731+0000 7f6efe605640 1 --2- 172.21.15.52:0/258899580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ef80cc450 0x7f6ef814a880 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.731+0000 7f6efe605640 1 -- 172.21.15.52:0/258899580 >> 172.21.15.52:0/258899580 conn(0x7f6ef80672a0 msgr2=0x7f6ef80afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.731+0000 7f6efe605640 1 -- 172.21.15.52:0/258899580 shutdown_connections 2024-05-04T21:20:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.731+0000 7f6efe605640 1 -- 172.21.15.52:0/258899580 wait complete. 2024-05-04T21:20:19.762 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T21:20:19.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.919+0000 7f015233d640 1 Processor -- start 2024-05-04T21:20:19.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.919+0000 7f015233d640 1 -- start start 2024-05-04T21:20:19.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.919+0000 7f015233d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f014c0c4280 0x7f014c15b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:19.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.919+0000 7f015233d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f014c15e970 0x7f014c06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f014b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f014c0c4280 0x7f014c15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f01508da640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f014c15e970 0x7f014c06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f014b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f014c0c4280 0x7f014c15b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:47776/0 (socket says 172.21.15.52:47776) 2024-05-04T21:20:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f01508da640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f014c15e970 0x7f014c06e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36818/0 (socket says 172.21.15.52:36818) 2024-05-04T21:20:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f014bfff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f014c0b2df0 0x7f014c0c3b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:45102/0 (socket says 172.21.15.52:45102) 2024-05-04T21:20:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f015233d640 1 -- 172.21.15.52:0/1701422502 --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f014c15bad0 con 0x7f014c0b2df0 2024-05-04T21:20:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f014b7fe640 1 -- 172.21.15.52:0/1701422502 learned_addr learned my addr 172.21.15.52:0/1701422502 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f015233d640 1 -- 172.21.15.52:0/1701422502 --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f014c0cd950 con 0x7f014c15e970 2024-05-04T21:20:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f015233d640 1 -- 172.21.15.52:0/1701422502 --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f014c15c370 con 0x7f014c0c4280 2024-05-04T21:20:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f014affd640 1 -- 172.21.15.52:0/1701422502 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3017569721 0 0) 0x7f014c15bad0 con 0x7f014c0b2df0 2024-05-04T21:20:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f014affd640 1 -- 172.21.15.52:0/1701422502 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0134007940 con 0x7f014c0b2df0 2024-05-04T21:20:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f01508da640 1 -- 172.21.15.52:0/1701422502 >> v2:172.21.15.111:3300/0 conn(0x7f014c0c4280 msgr2=0x7f014c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f01508da640 1 --2- 172.21.15.52:0/1701422502 >> v2:172.21.15.111:3300/0 conn(0x7f014c0c4280 0x7f014c15b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f01508da640 1 -- 172.21.15.52:0/1701422502 >> v1:172.21.15.52:6789/0 conn(0x7f014c0b2df0 legacy=0x7f014c0c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.923+0000 7f01508da640 1 -- 172.21.15.52:0/1701422502 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c15bdb0 con 0x7f014c15e970 2024-05-04T21:20:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.927+0000 7f01508da640 1 --2- 172.21.15.52:0/1701422502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f014c15e970 0x7f014c06e3b0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f014c06f980 tx=0x7f01440250b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447df2513f68cb44 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.927+0000 7f014affd640 1 -- 172.21.15.52:0/1701422502 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01440b9070 con 0x7f014c15e970 2024-05-04T21:20:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.927+0000 7f014affd640 1 -- 172.21.15.52:0/1701422502 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f014400b2d0 con 0x7f014c15e970 2024-05-04T21:20:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.927+0000 7f014affd640 1 -- 172.21.15.52:0/1701422502 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144009740 con 0x7f014c15e970 2024-05-04T21:20:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.927+0000 7f015233d640 1 -- 172.21.15.52:0/1701422502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f014c15e970 msgr2=0x7f014c06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.927+0000 7f015233d640 1 --2- 172.21.15.52:0/1701422502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f014c15e970 0x7f014c06e3b0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f014c06f980 tx=0x7f01440250b0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.927+0000 7f015233d640 1 -- 172.21.15.52:0/1701422502 shutdown_connections 2024-05-04T21:20:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.927+0000 7f015233d640 1 --2- 172.21.15.52:0/1701422502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f014c15e970 0x7f014c06e3b0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.927+0000 7f015233d640 1 --2- 172.21.15.52:0/1701422502 >> v2:172.21.15.111:3300/0 conn(0x7f014c0c4280 0x7f014c15b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.927+0000 7f015233d640 1 -- 172.21.15.52:0/1701422502 >> 172.21.15.52:0/1701422502 conn(0x7f014c0d12c0 msgr2=0x7f014c0d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.927+0000 7f015233d640 1 -- 172.21.15.52:0/1701422502 shutdown_connections 2024-05-04T21:20:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.927+0000 7f015233d640 1 -- 172.21.15.52:0/1701422502 wait complete. 2024-05-04T21:20:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.927+0000 7f015233d640 1 Processor -- start 2024-05-04T21:20:19.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.927+0000 7f015233d640 1 -- start start 2024-05-04T21:20:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f015233d640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014c0b2df0 0x7f014c143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f015233d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f014c0c4280 0x7f014c145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f015233d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f014c15e970 0x7f014c153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f015233d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f014c06f280 con 0x7f014c15e970 2024-05-04T21:20:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f015233d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f014c06f100 con 0x7f014c0c4280 2024-05-04T21:20:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f015233d640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f014c06f400 con 0x7f014c0b2df0 2024-05-04T21:20:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f01508da640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f014c15e970 0x7f014c153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f01508da640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f014c15e970 0x7f014c153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36832/0 (socket says 172.21.15.52:36832) 2024-05-04T21:20:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f01508da640 1 -- 172.21.15.52:0/1029739319 learned_addr learned my addr 172.21.15.52:0/1029739319 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f014b7fe640 1 --2- 172.21.15.52:0/1029739319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f014c0c4280 0x7f014c145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f014bfff640 1 --2- 172.21.15.52:0/1029739319 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014c0b2df0 0x7f014c143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f01508da640 1 -- 172.21.15.52:0/1029739319 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014c0b2df0 msgr2=0x7f014c143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f01508da640 1 --2- 172.21.15.52:0/1029739319 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014c0b2df0 0x7f014c143320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f01508da640 1 -- 172.21.15.52:0/1029739319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f014c0c4280 msgr2=0x7f014c145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f01508da640 1 --2- 172.21.15.52:0/1029739319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f014c0c4280 0x7f014c145dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f01508da640 1 -- 172.21.15.52:0/1029739319 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c153770 con 0x7f014c15e970 2024-05-04T21:20:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f01508da640 1 --2- 172.21.15.52:0/1029739319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f014c15e970 0x7f014c153230 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f0144022ce0 tx=0x7f0144021e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f0148ff9640 1 -- 172.21.15.52:0/1029739319 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01440c3070 con 0x7f014c15e970 2024-05-04T21:20:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f0148ff9640 1 -- 172.21.15.52:0/1029739319 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0144009c40 con 0x7f014c15e970 2024-05-04T21:20:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f015233d640 1 -- 172.21.15.52:0/1029739319 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014c147430 con 0x7f014c15e970 2024-05-04T21:20:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f0148ff9640 1 -- 172.21.15.52:0/1029739319 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01440b9040 con 0x7f014c15e970 2024-05-04T21:20:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f015233d640 1 -- 172.21.15.52:0/1029739319 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014c071170 con 0x7f014c15e970 2024-05-04T21:20:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.931+0000 7f015233d640 1 -- 172.21.15.52:0/1029739319 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014c147950 con 0x7f014c15e970 2024-05-04T21:20:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.939+0000 7f0148ff9640 1 -- 172.21.15.52:0/1029739319 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f01440bc070 con 0x7f014c15e970 2024-05-04T21:20:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.939+0000 7f0148ff9640 1 --2- 172.21.15.52:0/1029739319 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f01380678c0 0x7f0138069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.939+0000 7f0148ff9640 1 -- 172.21.15.52:0/1029739319 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0144128c20 con 0x7f014c15e970 2024-05-04T21:20:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.939+0000 7f014bfff640 1 --2- 172.21.15.52:0/1029739319 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f01380678c0 0x7f0138069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.939+0000 7f0148ff9640 1 -- 172.21.15.52:0/1029739319 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f014c147950 con 0x7f014c15e970 2024-05-04T21:20:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:19.939+0000 7f014bfff640 1 --2- 172.21.15.52:0/1029739319 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f01380678c0 0x7f0138069d80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f013c000f70 tx=0x7f013c007d00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:20.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.163+0000 7f015233d640 1 -- 172.21.15.52:0/1029739319 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f014c062e50 con 0x7f014c15e970 2024-05-04T21:20:20.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.163+0000 7f0148ff9640 1 -- 172.21.15.52:0/1029739319 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f014401ed30 con 0x7f014c15e970 2024-05-04T21:20:20.171 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-04T21:20:20.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.167+0000 7f015233d640 1 -- 172.21.15.52:0/1029739319 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f01380678c0 msgr2=0x7f0138069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:20.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.167+0000 7f015233d640 1 --2- 172.21.15.52:0/1029739319 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f01380678c0 0x7f0138069d80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f013c000f70 tx=0x7f013c007d00 comp rx=0 tx=0).stop 2024-05-04T21:20:20.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.167+0000 7f015233d640 1 -- 172.21.15.52:0/1029739319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f014c15e970 msgr2=0x7f014c153230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:20.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.167+0000 7f015233d640 1 --2- 172.21.15.52:0/1029739319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f014c15e970 0x7f014c153230 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f0144022ce0 tx=0x7f0144021e50 comp rx=0 tx=0).stop 2024-05-04T21:20:20.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.167+0000 7f015233d640 1 -- 172.21.15.52:0/1029739319 shutdown_connections 2024-05-04T21:20:20.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.167+0000 7f015233d640 1 --2- 172.21.15.52:0/1029739319 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f01380678c0 0x7f0138069d80 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.167+0000 7f015233d640 1 --2- 172.21.15.52:0/1029739319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f014c15e970 0x7f014c153230 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.167+0000 7f015233d640 1 --2- 172.21.15.52:0/1029739319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f014c0c4280 0x7f014c145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.167+0000 7f015233d640 1 --2- 172.21.15.52:0/1029739319 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014c0b2df0 0x7f014c143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.167+0000 7f015233d640 1 -- 172.21.15.52:0/1029739319 >> 172.21.15.52:0/1029739319 conn(0x7f014c0d12c0 msgr2=0x7f014c0c3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:20.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.167+0000 7f015233d640 1 -- 172.21.15.52:0/1029739319 shutdown_connections 2024-05-04T21:20:20.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.167+0000 7f015233d640 1 -- 172.21.15.52:0/1029739319 wait complete. 2024-05-04T21:20:20.192 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T21:20:20.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.347+0000 7f997b0cb640 1 Processor -- start 2024-05-04T21:20:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.347+0000 7f997b0cb640 1 -- start start 2024-05-04T21:20:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.347+0000 7f997b0cb640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f997415ed70 0x7f99740d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.347+0000 7f997b0cb640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99740cf350 con 0x7f997415aae0 2024-05-04T21:20:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.347+0000 7f997b0cb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99740cf630 con 0x7f99740ced90 2024-05-04T21:20:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.347+0000 7f997b0cb640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f99740d0930 con 0x7f997415ed70 2024-05-04T21:20:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.347+0000 7f9978e67640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f997415aae0 0x7f99740cc050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:45118/0 (socket says 172.21.15.52:45118) 2024-05-04T21:20:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.347+0000 7f9978e67640 1 -- 172.21.15.52:0/4110193353 learned_addr learned my addr 172.21.15.52:0/4110193353 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.347+0000 7f9979668640 1 --2- 172.21.15.52:0/4110193353 >> v2:172.21.15.52:3300/0 conn(0x7f997415ed70 0x7f99740d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2900869858 0 0) 0x7f99740cf630 con 0x7f99740ced90 2024-05-04T21:20:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9964003450 con 0x7f99740ced90 2024-05-04T21:20:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 641079545 0 0) 0x7f99740cf350 con 0x7f997415aae0 2024-05-04T21:20:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99740cf630 con 0x7f997415aae0 2024-05-04T21:20:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1404797944 0 0) 0x7f99740cf630 con 0x7f997415aae0 2024-05-04T21:20:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 >> v2:172.21.15.52:3300/0 conn(0x7f997415ed70 msgr2=0x7f99740d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 --2- 172.21.15.52:0/4110193353 >> v2:172.21.15.52:3300/0 conn(0x7f997415ed70 0x7f99740d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 >> v1:172.21.15.111:6789/0 conn(0x7f99740ced90 legacy=0x7f997415e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99740cf910 con 0x7f997415aae0 2024-05-04T21:20:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f9960001680 con 0x7f997415aae0 2024-05-04T21:20:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 >> v1:172.21.15.52:6789/0 conn(0x7f997415aae0 legacy=0x7f99740cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 --2- 172.21.15.52:0/4110193353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9964003c70 0x7f9964008860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 --2- 172.21.15.52:0/4110193353 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9964004130 0x7f996400ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 --2- 172.21.15.52:0/4110193353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f996400b480 0x7f996400d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f99740d0930 con 0x7f9964003c70 2024-05-04T21:20:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f996400e180 con 0x7f996400b480 2024-05-04T21:20:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f996400e380 con 0x7f9964004130 2024-05-04T21:20:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f9978e67640 1 --2- 172.21.15.52:0/4110193353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9964003c70 0x7f9964008860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f9973fff640 1 --2- 172.21.15.52:0/4110193353 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9964004130 0x7f996400ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f9978e67640 1 -- 172.21.15.52:0/4110193353 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9964004130 msgr2=0x7f996400ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f9978e67640 1 --2- 172.21.15.52:0/4110193353 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9964004130 0x7f996400ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f9978e67640 1 -- 172.21.15.52:0/4110193353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f996400b480 msgr2=0x7f996400d940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T21:20:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f9978e67640 1 --2- 172.21.15.52:0/4110193353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f996400b480 0x7f996400d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f9978e67640 1 -- 172.21.15.52:0/4110193353 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99740cf630 con 0x7f9964003c70 2024-05-04T21:20:20.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f9978e67640 1 --2- 172.21.15.52:0/4110193353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9964003c70 0x7f9964008860 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f996000c210 tx=0x7f996001f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0caadf2d1ad50a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:20.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960007cf0 con 0x7f9964003c70 2024-05-04T21:20:20.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9960005ae0 con 0x7f9964003c70 2024-05-04T21:20:20.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.351+0000 7f99737fe640 1 -- 172.21.15.52:0/4110193353 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960007070 con 0x7f9964003c70 2024-05-04T21:20:20.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.355+0000 7f9979668640 1 -- 172.21.15.52:0/4110193353 reap_dead start 2024-05-04T21:20:20.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.355+0000 7f997b0cb640 1 -- 172.21.15.52:0/4110193353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9964003c70 msgr2=0x7f9964008860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:20.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.355+0000 7f997b0cb640 1 --2- 172.21.15.52:0/4110193353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9964003c70 0x7f9964008860 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f996000c210 tx=0x7f996001f390 comp rx=0 tx=0).stop 2024-05-04T21:20:20.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.363+0000 7f997b0cb640 1 -- 172.21.15.52:0/4110193353 shutdown_connections 2024-05-04T21:20:20.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.363+0000 7f997b0cb640 1 --2- 172.21.15.52:0/4110193353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9964003c70 0x7f9964008860 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.363+0000 7f997b0cb640 1 -- 172.21.15.52:0/4110193353 >> 172.21.15.52:0/4110193353 conn(0x7f99740cd5b0 msgr2=0x7f99740cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:20.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.367+0000 7f997b0cb640 1 -- 172.21.15.52:0/4110193353 shutdown_connections 2024-05-04T21:20:20.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.371+0000 7f997b0cb640 1 -- 172.21.15.52:0/4110193353 wait complete. 2024-05-04T21:20:20.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.371+0000 7f997b0cb640 1 Processor -- start 2024-05-04T21:20:20.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.375+0000 7f997b0cb640 1 -- start start 2024-05-04T21:20:20.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f997b0cb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9964003c70 0x7f99741f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:20.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f997b0cb640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f997415d7b0 0x7f9974152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:20.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f997b0cb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99740c9c90 0x7f99741e4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:20.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f997b0cb640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f997415aec0 con 0x7f99740c9c90 2024-05-04T21:20:20.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f997b0cb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f99740cff60 con 0x7f9964003c70 2024-05-04T21:20:20.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f997b0cb640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f99740d03f0 con 0x7f997415d7b0 2024-05-04T21:20:20.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f9978e67640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9964003c70 0x7f99741f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:20.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f9978e67640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9964003c70 0x7f99741f8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:47782/0 (socket says 172.21.15.52:47782) 2024-05-04T21:20:20.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f9978e67640 1 -- 172.21.15.52:0/3244059622 learned_addr learned my addr 172.21.15.52:0/3244059622 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:20.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f9978e67640 1 -- 172.21.15.52:0/3244059622 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f997415d7b0 msgr2=0x7f9974152350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T21:20:20.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f9978e67640 1 --2- 172.21.15.52:0/3244059622 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f997415d7b0 0x7f9974152350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f9978e67640 1 -- 172.21.15.52:0/3244059622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99740c9c90 msgr2=0x7f99741e4390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T21:20:20.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f9978e67640 1 --2- 172.21.15.52:0/3244059622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99740c9c90 0x7f99741e4390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f9978e67640 1 -- 172.21.15.52:0/3244059622 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997415dc10 con 0x7f9964003c70 2024-05-04T21:20:20.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f9978e67640 1 --2- 172.21.15.52:0/3244059622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9964003c70 0x7f99741f8f00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f996001f1c0 tx=0x7f9964003740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:20.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f99717fa640 1 -- 172.21.15.52:0/3244059622 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960007cf0 con 0x7f9964003c70 2024-05-04T21:20:20.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f99717fa640 1 -- 172.21.15.52:0/3244059622 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99600076b0 con 0x7f9964003c70 2024-05-04T21:20:20.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f99717fa640 1 -- 172.21.15.52:0/3244059622 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960001c00 con 0x7f9964003c70 2024-05-04T21:20:20.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f997b0cb640 1 -- 172.21.15.52:0/3244059622 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99740c8050 con 0x7f9964003c70 2024-05-04T21:20:20.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f997b0cb640 1 -- 172.21.15.52:0/3244059622 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99740c8610 con 0x7f9964003c70 2024-05-04T21:20:20.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f997b0cb640 1 -- 172.21.15.52:0/3244059622 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9974062e50 con 0x7f9964003c70 2024-05-04T21:20:20.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f99717fa640 1 -- 172.21.15.52:0/3244059622 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f99600047f0 con 0x7f9964003c70 2024-05-04T21:20:20.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f99717fa640 1 --2- 172.21.15.52:0/3244059622 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f99580788d0 0x7f995807ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:20.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.379+0000 7f99717fa640 1 -- 172.21.15.52:0/3244059622 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f9960123120 con 0x7f9964003c70 2024-05-04T21:20:20.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.387+0000 7f9973fff640 1 --2- 172.21.15.52:0/3244059622 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f99580788d0 0x7f995807ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:20.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.387+0000 7f9973fff640 1 --2- 172.21.15.52:0/3244059622 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f99580788d0 0x7f995807ad90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f99740b17b0 tx=0x7f9968009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:20.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.387+0000 7f99717fa640 1 -- 172.21.15.52:0/3244059622 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f99600190a0 con 0x7f9964003c70 2024-05-04T21:20:20.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.663+0000 7f997b0cb640 1 -- 172.21.15.52:0/3244059622 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9974075320 con 0x7f9964003c70 2024-05-04T21:20:20.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.671+0000 7f99717fa640 1 -- 172.21.15.52:0/3244059622 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5184 (secure 0 0 0) 0x7f99600f3bc0 con 0x7f9964003c70 2024-05-04T21:20:20.677 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:20.677 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2438654614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2438654614},{"type":"v1","addr":"172.21.15.111:6835","nonce":2438654614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/796226117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":796226117},{"type":"v1","addr":"172.21.15.52:6835","nonce":796226117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3866938727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3866938727},{"type":"v1","addr":"172.21.15.111:6833","nonce":3866938727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-04T12:19:22.017880+0000","modified":"2024-05-04T12:19:25.052316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5199":{"gid":5199,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6837/1620779861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1620779861},{"type":"v1","addr":"172.21.15.52:6837","nonce":1620779861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5199,"qdb_cluster":[5199]},"id":2}]} 2024-05-04T21:20:20.677 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 16 2024-05-04T21:20:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.675+0000 7f997b0cb640 1 -- 172.21.15.52:0/3244059622 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f99580788d0 msgr2=0x7f995807ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.675+0000 7f997b0cb640 1 --2- 172.21.15.52:0/3244059622 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f99580788d0 0x7f995807ad90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f99740b17b0 tx=0x7f9968009040 comp rx=0 tx=0).stop 2024-05-04T21:20:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.675+0000 7f997b0cb640 1 -- 172.21.15.52:0/3244059622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9964003c70 msgr2=0x7f99741f8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.675+0000 7f997b0cb640 1 --2- 172.21.15.52:0/3244059622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9964003c70 0x7f99741f8f00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f996001f1c0 tx=0x7f9964003740 comp rx=0 tx=0).stop 2024-05-04T21:20:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.675+0000 7f997b0cb640 1 -- 172.21.15.52:0/3244059622 shutdown_connections 2024-05-04T21:20:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.675+0000 7f997b0cb640 1 --2- 172.21.15.52:0/3244059622 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f99580788d0 0x7f995807ad90 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.675+0000 7f997b0cb640 1 --2- 172.21.15.52:0/3244059622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99740c9c90 0x7f99741e4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.675+0000 7f997b0cb640 1 --2- 172.21.15.52:0/3244059622 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f997415d7b0 0x7f9974152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.675+0000 7f997b0cb640 1 --2- 172.21.15.52:0/3244059622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9964003c70 0x7f99741f8f00 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.675+0000 7f997b0cb640 1 -- 172.21.15.52:0/3244059622 >> 172.21.15.52:0/3244059622 conn(0x7f99740cd5b0 msgr2=0x7f9974159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:20.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.675+0000 7f997b0cb640 1 -- 172.21.15.52:0/3244059622 shutdown_connections 2024-05-04T21:20:20.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.675+0000 7f997b0cb640 1 -- 172.21.15.52:0/3244059622 wait complete. 2024-05-04T21:20:20.704 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T21:20:20.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.831+0000 7f92eab19640 1 Processor -- start 2024-05-04T21:20:20.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.831+0000 7f92eab19640 1 -- start start 2024-05-04T21:20:20.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92eab19640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f92e415c000 0x7f92e40ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:20.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92e3fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f92e415d360 0x7f92e40a9480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:45122/0 (socket says 172.21.15.52:45122) 2024-05-04T21:20:20.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92e3fff640 1 -- 172.21.15.52:0/3542630107 learned_addr learned my addr 172.21.15.52:0/3542630107 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:20.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92e88b5640 1 --2- 172.21.15.52:0/3542630107 >> v2:172.21.15.111:3300/0 conn(0x7f92e415c000 0x7f92e40ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:20.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92eab19640 1 -- 172.21.15.52:0/3542630107 --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92e40cd510 con 0x7f92e415d360 2024-05-04T21:20:20.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92eab19640 1 -- 172.21.15.52:0/3542630107 --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92e40cd7f0 con 0x7f92e4071b90 2024-05-04T21:20:20.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92e37fe640 1 -- 172.21.15.52:0/3542630107 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1106047126 0 0) 0x7f92e40cd510 con 0x7f92e415d360 2024-05-04T21:20:20.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92eab19640 1 -- 172.21.15.52:0/3542630107 --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f92e40cdad0 con 0x7f92e415c000 2024-05-04T21:20:20.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92e37fe640 1 -- 172.21.15.52:0/3542630107 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92d4003530 con 0x7f92e415d360 2024-05-04T21:20:20.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92e37fe640 1 -- 172.21.15.52:0/3542630107 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3451151193 0 0) 0x7f92e40cd7f0 con 0x7f92e4071b90 2024-05-04T21:20:20.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92e37fe640 1 -- 172.21.15.52:0/3542630107 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92e40cd510 con 0x7f92e4071b90 2024-05-04T21:20:20.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92e88b5640 1 -- 172.21.15.52:0/3542630107 >> v1:172.21.15.111:6789/0 conn(0x7f92e4071b90 legacy=0x7f92e40b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:20.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92e37fe640 1 -- 172.21.15.52:0/3542630107 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1737144945 0 0) 0x7f92d4003530 con 0x7f92e415d360 2024-05-04T21:20:20.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92e88b5640 1 -- 172.21.15.52:0/3542630107 >> v1:172.21.15.52:6789/0 conn(0x7f92e415d360 legacy=0x7f92e40a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:20.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.835+0000 7f92e88b5640 1 -- 172.21.15.52:0/3542630107 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e4071390 con 0x7f92e415c000 2024-05-04T21:20:20.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.839+0000 7f92e88b5640 1 --2- 172.21.15.52:0/3542630107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92e415c000 0x7f92e40ccba0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f92d00097c0 tx=0x7f92d001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26fa15e6bd7ff368 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:20.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.839+0000 7f92e37fe640 1 -- 172.21.15.52:0/3542630107 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d00b0070 con 0x7f92e415c000 2024-05-04T21:20:20.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.839+0000 7f92e37fe640 1 -- 172.21.15.52:0/3542630107 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d00ab400 con 0x7f92e415c000 2024-05-04T21:20:20.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.839+0000 7f92e37fe640 1 -- 172.21.15.52:0/3542630107 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8001680 con 0x7f92e415c000 2024-05-04T21:20:20.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.843+0000 7f92eab19640 1 -- 172.21.15.52:0/3542630107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92e415c000 msgr2=0x7f92e40ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:20.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.843+0000 7f92eab19640 1 --2- 172.21.15.52:0/3542630107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92e415c000 0x7f92e40ccba0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f92d00097c0 tx=0x7f92d001c3c0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.859+0000 7f92eab19640 1 -- 172.21.15.52:0/3542630107 shutdown_connections 2024-05-04T21:20:20.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.859+0000 7f92eab19640 1 --2- 172.21.15.52:0/3542630107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92e415c000 0x7f92e40ccba0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.859+0000 7f92eab19640 1 -- 172.21.15.52:0/3542630107 >> 172.21.15.52:0/3542630107 conn(0x7f92e4067070 msgr2=0x7f92e4067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:20.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.859+0000 7f92eab19640 1 -- 172.21.15.52:0/3542630107 shutdown_connections 2024-05-04T21:20:20.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92eab19640 1 -- 172.21.15.52:0/3542630107 wait complete. 2024-05-04T21:20:20.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92eab19640 1 Processor -- start 2024-05-04T21:20:20.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92eab19640 1 -- start start 2024-05-04T21:20:20.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92eab19640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92e4071b90 0x7f92e414ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:20.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92eab19640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92e415c000 0x7f92e4150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:20.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92eab19640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92e415d360 0x7f92e41433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:20.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92eab19640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f92e40b37f0 con 0x7f92e4071b90 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92eab19640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f92e40b3670 con 0x7f92e415c000 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92eab19640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f92e4071fa0 con 0x7f92e415d360 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92e88b5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92e4071b90 0x7f92e414ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92e88b5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92e4071b90 0x7f92e414ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36852/0 (socket says 172.21.15.52:36852) 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92e88b5640 1 -- 172.21.15.52:0/1946370505 learned_addr learned my addr 172.21.15.52:0/1946370505 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92e90b6640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92e415d360 0x7f92e41433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92e3fff640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92e415c000 0x7f92e4150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92e88b5640 1 -- 172.21.15.52:0/1946370505 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92e415d360 msgr2=0x7f92e41433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92e88b5640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92e415d360 0x7f92e41433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92e88b5640 1 -- 172.21.15.52:0/1946370505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92e415c000 msgr2=0x7f92e4150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92e88b5640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92e415c000 0x7f92e4150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92e88b5640 1 -- 172.21.15.52:0/1946370505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e4151f90 con 0x7f92e4071b90 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.863+0000 7f92e90b6640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92e415d360 0x7f92e41433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.867+0000 7f92e88b5640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92e4071b90 0x7f92e414ce10 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f92d00ae040 tx=0x7f92d001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.867+0000 7f92e17fa640 1 -- 172.21.15.52:0/1946370505 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d00bb020 con 0x7f92e4071b90 2024-05-04T21:20:20.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.867+0000 7f92e17fa640 1 -- 172.21.15.52:0/1946370505 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d00b1040 con 0x7f92e4071b90 2024-05-04T21:20:20.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.867+0000 7f92eab19640 1 -- 172.21.15.52:0/1946370505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e4152270 con 0x7f92e4071b90 2024-05-04T21:20:20.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.867+0000 7f92e17fa640 1 -- 172.21.15.52:0/1946370505 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d00b0050 con 0x7f92e4071b90 2024-05-04T21:20:20.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.867+0000 7f92eab19640 1 -- 172.21.15.52:0/1946370505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e4150410 con 0x7f92e4071b90 2024-05-04T21:20:20.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.867+0000 7f92eab19640 1 -- 172.21.15.52:0/1946370505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e4148c50 con 0x7f92e4071b90 2024-05-04T21:20:20.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.867+0000 7f92e17fa640 1 -- 172.21.15.52:0/1946370505 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f92d00be070 con 0x7f92e4071b90 2024-05-04T21:20:20.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.867+0000 7f92e17fa640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f92c80678f0 0x7f92c8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:20.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.867+0000 7f92e17fa640 1 -- 172.21.15.52:0/1946370505 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f92d011f2f0 con 0x7f92e4071b90 2024-05-04T21:20:20.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.867+0000 7f92e3fff640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f92c80678f0 0x7f92c8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:20.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.867+0000 7f92e3fff640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f92c80678f0 0x7f92c8069db0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f92e4152550 tx=0x7f92d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:20.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:20.871+0000 7f92e17fa640 1 -- 172.21.15.52:0/1946370505 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f92d0015fb0 con 0x7f92e4071b90 2024-05-04T21:20:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.127+0000 7f92eab19640 1 -- 172.21.15.52:0/1946370505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92e4062e50 con 0x7f92e4071b90 2024-05-04T21:20:21.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.163+0000 7f92e17fa640 1 -- 172.21.15.52:0/1946370505 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16681 (secure 0 0 0) 0x7f92d00b3300 con 0x7f92e4071b90 2024-05-04T21:20:21.170 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:21.170 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":34,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:19:23.639019+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:19:20.589701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-04T12:19:20.997186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T21:20:21.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.167+0000 7f92eab19640 1 -- 172.21.15.52:0/1946370505 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f92c80678f0 msgr2=0x7f92c8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:21.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.167+0000 7f92eab19640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f92c80678f0 0x7f92c8069db0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f92e4152550 tx=0x7f92d8009040 comp rx=0 tx=0).stop 2024-05-04T21:20:21.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.167+0000 7f92eab19640 1 -- 172.21.15.52:0/1946370505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92e4071b90 msgr2=0x7f92e414ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:21.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.167+0000 7f92eab19640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92e4071b90 0x7f92e414ce10 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f92d00ae040 tx=0x7f92d001ac00 comp rx=0 tx=0).stop 2024-05-04T21:20:21.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.167+0000 7f92eab19640 1 -- 172.21.15.52:0/1946370505 shutdown_connections 2024-05-04T21:20:21.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.167+0000 7f92eab19640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f92c80678f0 0x7f92c8069db0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.167+0000 7f92eab19640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92e415d360 0x7f92e41433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.167+0000 7f92eab19640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92e415c000 0x7f92e4150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.167+0000 7f92eab19640 1 --2- 172.21.15.52:0/1946370505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92e4071b90 0x7f92e414ce10 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.167+0000 7f92eab19640 1 -- 172.21.15.52:0/1946370505 >> 172.21.15.52:0/1946370505 conn(0x7f92e4067070 msgr2=0x7f92e40a9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.167+0000 7f92eab19640 1 -- 172.21.15.52:0/1946370505 shutdown_connections 2024-05-04T21:20:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.167+0000 7f92eab19640 1 -- 172.21.15.52:0/1946370505 wait complete. 2024-05-04T21:20:21.215 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-04T21:20:21.216 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T21:20:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.363+0000 7fc90d8b1640 1 Processor -- start 2024-05-04T21:20:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.363+0000 7fc90d8b1640 1 -- start start 2024-05-04T21:20:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.363+0000 7fc90d8b1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc90806da80 0x7fc908067070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.363+0000 7fc90d8b1640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc90806f6e0 0x7fc9080cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.363+0000 7fc90d8b1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9080ce7b0 con 0x7fc9080cfd90 2024-05-04T21:20:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.363+0000 7fc90d8b1640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fc9080cf3c0 con 0x7fc90806da80 2024-05-04T21:20:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.363+0000 7fc90d8b1640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc9080cea90 con 0x7fc90806f6e0 2024-05-04T21:20:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.363+0000 7fc906ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc90806da80 0x7fc908067070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.363+0000 7fc906ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc90806da80 0x7fc908067070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:47814/0 (socket says 172.21.15.52:47814) 2024-05-04T21:20:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.363+0000 7fc906ffd640 1 -- 172.21.15.52:0/3413528754 learned_addr learned my addr 172.21.15.52:0/3413528754 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.363+0000 7fc906ffd640 1 -- 172.21.15.52:0/3413528754 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc90806f6e0 msgr2=0x7fc9080cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.363+0000 7fc906ffd640 1 --2- 172.21.15.52:0/3413528754 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc90806f6e0 0x7fc9080cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.363+0000 7fc906ffd640 1 -- 172.21.15.52:0/3413528754 >> v1:172.21.15.52:6789/0 conn(0x7fc9080cfd90 legacy=0x7fc908161e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.363+0000 7fc906ffd640 1 -- 172.21.15.52:0/3413528754 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9080ced70 con 0x7fc90806da80 2024-05-04T21:20:21.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.367+0000 7fc906ffd640 1 --2- 172.21.15.52:0/3413528754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc90806da80 0x7fc908067070 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009870 tx=0x7fc8f801c570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aae5bd8cf67a948 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:21.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.367+0000 7fc9067fc640 1 -- 172.21.15.52:0/3413528754 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80a9650 con 0x7fc90806da80 2024-05-04T21:20:21.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.367+0000 7fc9067fc640 1 -- 172.21.15.52:0/3413528754 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8f80a97f0 con 0x7fc90806da80 2024-05-04T21:20:21.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.367+0000 7fc9067fc640 1 -- 172.21.15.52:0/3413528754 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80a9c00 con 0x7fc90806da80 2024-05-04T21:20:21.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.371+0000 7fc90d8b1640 1 -- 172.21.15.52:0/3413528754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc90806da80 msgr2=0x7fc908067070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:21.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.371+0000 7fc90d8b1640 1 --2- 172.21.15.52:0/3413528754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc90806da80 0x7fc908067070 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009870 tx=0x7fc8f801c570 comp rx=0 tx=0).stop 2024-05-04T21:20:21.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.371+0000 7fc90d8b1640 1 -- 172.21.15.52:0/3413528754 shutdown_connections 2024-05-04T21:20:21.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.371+0000 7fc90d8b1640 1 --2- 172.21.15.52:0/3413528754 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc90806f6e0 0x7fc9080cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.371+0000 7fc90d8b1640 1 --2- 172.21.15.52:0/3413528754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc90806da80 0x7fc908067070 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.371+0000 7fc90d8b1640 1 -- 172.21.15.52:0/3413528754 >> 172.21.15.52:0/3413528754 conn(0x7fc9080b2fe0 msgr2=0x7fc9080b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.371+0000 7fc90d8b1640 1 -- 172.21.15.52:0/3413528754 shutdown_connections 2024-05-04T21:20:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.371+0000 7fc90d8b1640 1 -- 172.21.15.52:0/3413528754 wait complete. 2024-05-04T21:20:21.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc90d8b1640 1 Processor -- start 2024-05-04T21:20:21.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc90d8b1640 1 -- start start 2024-05-04T21:20:21.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc90d8b1640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc90806da80 0x7fc90812f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:21.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc90d8b1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc90806f6e0 0x7fc908146cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:21.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc90d8b1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc90812e070 0x7fc9081609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:21.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc90d8b1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc908162550 con 0x7fc90812e070 2024-05-04T21:20:21.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc90d8b1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc9080ae2b0 con 0x7fc90806f6e0 2024-05-04T21:20:21.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc90d8b1640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc9081626d0 con 0x7fc90806da80 2024-05-04T21:20:21.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc906ffd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc90806da80 0x7fc90812f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:21.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc906ffd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc90806da80 0x7fc90812f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.52:35292/0 (socket says 172.21.15.52:35292) 2024-05-04T21:20:21.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc906ffd640 1 -- 172.21.15.52:0/1404670107 learned_addr learned my addr 172.21.15.52:0/1404670107 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:21.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc9077fe640 1 --2- 172.21.15.52:0/1404670107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc90812e070 0x7fc9081609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:21.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc9077fe640 1 -- 172.21.15.52:0/1404670107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc90806da80 msgr2=0x7fc90812f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:21.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc9077fe640 1 --2- 172.21.15.52:0/1404670107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc90806da80 0x7fc90812f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc9077fe640 1 -- 172.21.15.52:0/1404670107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc90806f6e0 msgr2=0x7fc908146cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:21.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc9077fe640 1 --2- 172.21.15.52:0/1404670107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc90806f6e0 0x7fc908146cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc9077fe640 1 -- 172.21.15.52:0/1404670107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc908147420 con 0x7fc90812e070 2024-05-04T21:20:21.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc9077fe640 1 --2- 172.21.15.52:0/1404670107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc90812e070 0x7fc9081609d0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fc908147fb0 tx=0x7fc90000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:21.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc904ff9640 1 -- 172.21.15.52:0/1404670107 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900016020 con 0x7fc90812e070 2024-05-04T21:20:21.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc904ff9640 1 -- 172.21.15.52:0/1404670107 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc900006d60 con 0x7fc90812e070 2024-05-04T21:20:21.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.375+0000 7fc904ff9640 1 -- 172.21.15.52:0/1404670107 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900003bc0 con 0x7fc90812e070 2024-05-04T21:20:21.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.379+0000 7fc90d8b1640 1 -- 172.21.15.52:0/1404670107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc908147700 con 0x7fc90812e070 2024-05-04T21:20:21.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.379+0000 7fc90d8b1640 1 -- 172.21.15.52:0/1404670107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9080ae960 con 0x7fc90812e070 2024-05-04T21:20:21.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.379+0000 7fc8fdffb640 1 -- 172.21.15.52:0/1404670107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e00052a0 con 0x7fc90812e070 2024-05-04T21:20:21.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.379+0000 7fc904ff9640 1 -- 172.21.15.52:0/1404670107 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc9000050d0 con 0x7fc90812e070 2024-05-04T21:20:21.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.379+0000 7fc904ff9640 1 --2- 172.21.15.52:0/1404670107 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc8e80678c0 0x7fc8e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:21.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.379+0000 7fc904ff9640 1 -- 172.21.15.52:0/1404670107 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc900085690 con 0x7fc90812e070 2024-05-04T21:20:21.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.379+0000 7fc906ffd640 1 --2- 172.21.15.52:0/1404670107 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc8e80678c0 0x7fc8e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:21.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.379+0000 7fc906ffd640 1 --2- 172.21.15.52:0/1404670107 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc8e80678c0 0x7fc8e8069d80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8000c10 tx=0x7fc8f80b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:21.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.383+0000 7fc904ff9640 1 -- 172.21.15.52:0/1404670107 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc900020020 con 0x7fc90812e070 2024-05-04T21:20:21.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.675+0000 7fc8fdffb640 1 -- 172.21.15.52:0/1404670107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8e0005860 con 0x7fc90812e070 2024-05-04T21:20:21.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.675+0000 7fc904ff9640 1 -- 172.21.15.52:0/1404670107 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5184 (secure 0 0 0) 0x7fc900056260 con 0x7fc90812e070 2024-05-04T21:20:21.679 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:21.679 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2438654614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2438654614},{"type":"v1","addr":"172.21.15.111:6835","nonce":2438654614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/796226117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":796226117},{"type":"v1","addr":"172.21.15.52:6835","nonce":796226117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3866938727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3866938727},{"type":"v1","addr":"172.21.15.111:6833","nonce":3866938727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-04T12:19:22.017880+0000","modified":"2024-05-04T12:19:25.052316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5199":{"gid":5199,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6837/1620779861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1620779861},{"type":"v1","addr":"172.21.15.52:6837","nonce":1620779861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5199,"qdb_cluster":[5199]},"id":2}]} 2024-05-04T21:20:21.680 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 16 2024-05-04T21:20:21.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.675+0000 7fc8fdffb640 1 -- 172.21.15.52:0/1404670107 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc8e80678c0 msgr2=0x7fc8e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:21.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.675+0000 7fc8fdffb640 1 --2- 172.21.15.52:0/1404670107 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc8e80678c0 0x7fc8e8069d80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8000c10 tx=0x7fc8f80b3040 comp rx=0 tx=0).stop 2024-05-04T21:20:21.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.675+0000 7fc8fdffb640 1 -- 172.21.15.52:0/1404670107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc90812e070 msgr2=0x7fc9081609d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:21.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.675+0000 7fc8fdffb640 1 --2- 172.21.15.52:0/1404670107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc90812e070 0x7fc9081609d0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fc908147fb0 tx=0x7fc90000cb90 comp rx=0 tx=0).stop 2024-05-04T21:20:21.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.675+0000 7fc8fdffb640 1 -- 172.21.15.52:0/1404670107 shutdown_connections 2024-05-04T21:20:21.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.675+0000 7fc8fdffb640 1 --2- 172.21.15.52:0/1404670107 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc8e80678c0 0x7fc8e8069d80 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.675+0000 7fc8fdffb640 1 --2- 172.21.15.52:0/1404670107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc90812e070 0x7fc9081609d0 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.675+0000 7fc8fdffb640 1 --2- 172.21.15.52:0/1404670107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc90806f6e0 0x7fc908146cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.675+0000 7fc8fdffb640 1 --2- 172.21.15.52:0/1404670107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc90806da80 0x7fc90812f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.675+0000 7fc8fdffb640 1 -- 172.21.15.52:0/1404670107 >> 172.21.15.52:0/1404670107 conn(0x7fc9080b2fe0 msgr2=0x7fc9080b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:21.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.675+0000 7fc8fdffb640 1 -- 172.21.15.52:0/1404670107 shutdown_connections 2024-05-04T21:20:21.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.675+0000 7fc8fdffb640 1 -- 172.21.15.52:0/1404670107 wait complete. 2024-05-04T21:20:21.705 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T21:20:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.843+0000 7fb5c7881640 1 Processor -- start 2024-05-04T21:20:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.847+0000 7fb5c7881640 1 -- start start 2024-05-04T21:20:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.847+0000 7fb5c7881640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb5c0070930 0x7fb5c01601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.847+0000 7fb5c7881640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb5c015c870 0x7fb5c015cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.847+0000 7fb5c7881640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c006dd90 con 0x7fb5c0070d10 2024-05-04T21:20:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.847+0000 7fb5c7881640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb5c006e000 con 0x7fb5c015c870 2024-05-04T21:20:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.847+0000 7fb5c7881640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fb5c006e2e0 con 0x7fb5c0070930 2024-05-04T21:20:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.847+0000 7fb5c5e1e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb5c015c870 0x7fb5c015cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.847+0000 7fb5c5e1e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb5c015c870 0x7fb5c015cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36876/0 (socket says 172.21.15.52:36876) 2024-05-04T21:20:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.847+0000 7fb5c5e1e640 1 -- 172.21.15.52:0/42002043 learned_addr learned my addr 172.21.15.52:0/42002043 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.847+0000 7fb5c561d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb5c0070930 0x7fb5c01601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.847+0000 7fb5c5e1e640 1 -- 172.21.15.52:0/42002043 >> v2:172.21.15.111:3300/0 conn(0x7fb5c0070930 msgr2=0x7fb5c01601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.847+0000 7fb5c5e1e640 1 --2- 172.21.15.52:0/42002043 >> v2:172.21.15.111:3300/0 conn(0x7fb5c0070930 0x7fb5c01601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.847+0000 7fb5c5e1e640 1 -- 172.21.15.52:0/42002043 >> v1:172.21.15.111:6789/0 conn(0x7fb5c0070d10 legacy=0x7fb5c0160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.847+0000 7fb5c5e1e640 1 -- 172.21.15.52:0/42002043 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c006e5c0 con 0x7fb5c015c870 2024-05-04T21:20:21.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.851+0000 7fb5c5e1e640 1 --2- 172.21.15.52:0/42002043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5c015c870 0x7fb5c015cd00 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0071400 tx=0x7fb5b801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2380e7bcb9d919b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:21.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.851+0000 7fb5b7fff640 1 -- 172.21.15.52:0/42002043 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b80a57e0 con 0x7fb5c015c870 2024-05-04T21:20:21.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.851+0000 7fb5b7fff640 1 -- 172.21.15.52:0/42002043 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b80a5980 con 0x7fb5c015c870 2024-05-04T21:20:21.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.851+0000 7fb5b7fff640 1 -- 172.21.15.52:0/42002043 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b80a5d50 con 0x7fb5c015c870 2024-05-04T21:20:21.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.851+0000 7fb5c7881640 1 -- 172.21.15.52:0/42002043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5c015c870 msgr2=0x7fb5c015cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:21.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.851+0000 7fb5c7881640 1 --2- 172.21.15.52:0/42002043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5c015c870 0x7fb5c015cd00 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0071400 tx=0x7fb5b801c9f0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.855+0000 7fb5c7881640 1 -- 172.21.15.52:0/42002043 shutdown_connections 2024-05-04T21:20:21.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.855+0000 7fb5c7881640 1 --2- 172.21.15.52:0/42002043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5c015c870 0x7fb5c015cd00 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.855+0000 7fb5c7881640 1 --2- 172.21.15.52:0/42002043 >> v2:172.21.15.111:3300/0 conn(0x7fb5c0070930 0x7fb5c01601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.855+0000 7fb5c7881640 1 -- 172.21.15.52:0/42002043 >> 172.21.15.52:0/42002043 conn(0x7fb5c0067070 msgr2=0x7fb5c0067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:21.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.855+0000 7fb5c7881640 1 -- 172.21.15.52:0/42002043 shutdown_connections 2024-05-04T21:20:21.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c7881640 1 -- 172.21.15.52:0/42002043 wait complete. 2024-05-04T21:20:21.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c7881640 1 Processor -- start 2024-05-04T21:20:21.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c7881640 1 -- start start 2024-05-04T21:20:21.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c7881640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5c0070930 0x7fb5c013d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:21.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c561d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5c0070930 0x7fb5c013d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:21.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c561d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5c0070930 0x7fb5c013d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36878/0 (socket says 172.21.15.52:36878) 2024-05-04T21:20:21.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c561d640 1 -- 172.21.15.52:0/3126388370 learned_addr learned my addr 172.21.15.52:0/3126388370 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:21.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c7881640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c0070d10 0x7fb5c015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:21.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c7881640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5c015c870 0x7fb5c015b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:21.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c7881640 1 -- 172.21.15.52:0/3126388370 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c0168a20 con 0x7fb5c0070930 2024-05-04T21:20:21.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c7881640 1 -- 172.21.15.52:0/3126388370 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c01688a0 con 0x7fb5c0070d10 2024-05-04T21:20:21.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c7881640 1 -- 172.21.15.52:0/3126388370 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c0168ba0 con 0x7fb5c015c870 2024-05-04T21:20:21.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c5e1e640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5c015c870 0x7fb5c015b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:21.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c561d640 1 -- 172.21.15.52:0/3126388370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5c015c870 msgr2=0x7fb5c015b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:21.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c561d640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5c015c870 0x7fb5c015b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c561d640 1 -- 172.21.15.52:0/3126388370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c0070d10 msgr2=0x7fb5c015b1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T21:20:21.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c561d640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c0070d10 0x7fb5c015b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:21.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c561d640 1 -- 172.21.15.52:0/3126388370 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c013e500 con 0x7fb5c0070930 2024-05-04T21:20:21.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c561d640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5c0070930 0x7fb5c013d080 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00e870 tx=0x7fb5bc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:21.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5b5ffb640 1 -- 172.21.15.52:0/3126388370 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc017070 con 0x7fb5c0070930 2024-05-04T21:20:21.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5b5ffb640 1 -- 172.21.15.52:0/3126388370 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5bc00c940 con 0x7fb5c0070930 2024-05-04T21:20:21.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5b5ffb640 1 -- 172.21.15.52:0/3126388370 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc00ccf0 con 0x7fb5c0070930 2024-05-04T21:20:21.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c7881640 1 -- 172.21.15.52:0/3126388370 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c013fdb0 con 0x7fb5c0070930 2024-05-04T21:20:21.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.859+0000 7fb5c7881640 1 -- 172.21.15.52:0/3126388370 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c013dc60 con 0x7fb5c0070930 2024-05-04T21:20:21.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.863+0000 7fb5c7881640 1 -- 172.21.15.52:0/3126388370 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c0062e50 con 0x7fb5c0070930 2024-05-04T21:20:21.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.863+0000 7fb5b5ffb640 1 -- 172.21.15.52:0/3126388370 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb5bc01a070 con 0x7fb5c0070930 2024-05-04T21:20:21.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.863+0000 7fb5b5ffb640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb5a40678c0 0x7fb5a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:21.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.863+0000 7fb5b5ffb640 1 -- 172.21.15.52:0/3126388370 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fb5bc085bc0 con 0x7fb5c0070930 2024-05-04T21:20:21.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.863+0000 7fb5c4e1c640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb5a40678c0 0x7fb5a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:21.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.863+0000 7fb5c4e1c640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb5a40678c0 0x7fb5a4069d80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb5c01394e0 tx=0x7fb5b00079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:21.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:21.871+0000 7fb5b5ffb640 1 -- 172.21.15.52:0/3126388370 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb5c013e500 con 0x7fb5c0070930 2024-05-04T21:20:22.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.147+0000 7fb5c7881640 1 -- 172.21.15.52:0/3126388370 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5c0071ee0 con 0x7fb5c0070930 2024-05-04T21:20:22.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.147+0000 7fb5b5ffb640 1 -- 172.21.15.52:0/3126388370 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5184 (secure 0 0 0) 0x7fb5bc056700 con 0x7fb5c0070930 2024-05-04T21:20:22.151 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:22.151 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2438654614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2438654614},{"type":"v1","addr":"172.21.15.111:6835","nonce":2438654614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/796226117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":796226117},{"type":"v1","addr":"172.21.15.52:6835","nonce":796226117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3866938727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3866938727},{"type":"v1","addr":"172.21.15.111:6833","nonce":3866938727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-04T12:19:22.017880+0000","modified":"2024-05-04T12:19:25.052316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5199":{"gid":5199,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6837/1620779861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1620779861},{"type":"v1","addr":"172.21.15.52:6837","nonce":1620779861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5199,"qdb_cluster":[5199]},"id":2}]} 2024-05-04T21:20:22.151 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 16 2024-05-04T21:20:22.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.147+0000 7fb5c7881640 1 -- 172.21.15.52:0/3126388370 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb5a40678c0 msgr2=0x7fb5a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:22.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.147+0000 7fb5c7881640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb5a40678c0 0x7fb5a4069d80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb5c01394e0 tx=0x7fb5b00079e0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.147+0000 7fb5c7881640 1 -- 172.21.15.52:0/3126388370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5c0070930 msgr2=0x7fb5c013d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:22.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.147+0000 7fb5c7881640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5c0070930 0x7fb5c013d080 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00e870 tx=0x7fb5bc00ec70 comp rx=0 tx=0).stop 2024-05-04T21:20:22.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.147+0000 7fb5c7881640 1 -- 172.21.15.52:0/3126388370 shutdown_connections 2024-05-04T21:20:22.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.147+0000 7fb5c7881640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb5a40678c0 0x7fb5a4069d80 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.147+0000 7fb5c7881640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5c015c870 0x7fb5c015b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.147+0000 7fb5c7881640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c0070d10 0x7fb5c015b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.147+0000 7fb5c7881640 1 --2- 172.21.15.52:0/3126388370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5c0070930 0x7fb5c013d080 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.147+0000 7fb5c7881640 1 -- 172.21.15.52:0/3126388370 >> 172.21.15.52:0/3126388370 conn(0x7fb5c0067070 msgr2=0x7fb5c00b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:22.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.147+0000 7fb5c7881640 1 -- 172.21.15.52:0/3126388370 shutdown_connections 2024-05-04T21:20:22.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.147+0000 7fb5c7881640 1 -- 172.21.15.52:0/3126388370 wait complete. 2024-05-04T21:20:22.189 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T21:20:22.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7faced4eb640 1 Processor -- start 2024-05-04T21:20:22.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7faced4eb640 1 -- start start 2024-05-04T21:20:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7faced4eb640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7face8148e00 0x7face8071210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7faced4eb640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7face80b31c0 0x7face80ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7faced4eb640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face80cd780 con 0x7face8148a20 2024-05-04T21:20:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7faced4eb640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7face80cda60 con 0x7face8148e00 2024-05-04T21:20:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7faced4eb640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7face8070730 con 0x7face80b31c0 2024-05-04T21:20:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7face67fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7face8148e00 0x7face8071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7face67fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7face8148e00 0x7face8071210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36886/0 (socket says 172.21.15.52:36886) 2024-05-04T21:20:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7face67fc640 1 -- 172.21.15.52:0/3483538066 learned_addr learned my addr 172.21.15.52:0/3483538066 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7face77fe640 1 --2- 172.21.15.52:0/3483538066 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7face80b31c0 0x7face80ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7face67fc640 1 -- 172.21.15.52:0/3483538066 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7face80b31c0 msgr2=0x7face80ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7face67fc640 1 --2- 172.21.15.52:0/3483538066 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7face80b31c0 0x7face80ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7face67fc640 1 -- 172.21.15.52:0/3483538066 >> v1:172.21.15.52:6789/0 conn(0x7face8148a20 legacy=0x7face80b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.323+0000 7face67fc640 1 -- 172.21.15.52:0/3483538066 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face8070a10 con 0x7face8148e00 2024-05-04T21:20:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.327+0000 7face67fc640 1 --2- 172.21.15.52:0/3483538066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7face8148e00 0x7face8071210 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7facdc009870 tx=0x7facdc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bedfcf63b4e7d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.327+0000 7face5ffb640 1 -- 172.21.15.52:0/3483538066 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc0a9610 con 0x7face8148e00 2024-05-04T21:20:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.327+0000 7face5ffb640 1 -- 172.21.15.52:0/3483538066 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facdc0a97b0 con 0x7face8148e00 2024-05-04T21:20:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.327+0000 7face5ffb640 1 -- 172.21.15.52:0/3483538066 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc0a9b80 con 0x7face8148e00 2024-05-04T21:20:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.327+0000 7faced4eb640 1 -- 172.21.15.52:0/3483538066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7face8148e00 msgr2=0x7face8071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.327+0000 7faced4eb640 1 --2- 172.21.15.52:0/3483538066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7face8148e00 0x7face8071210 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7facdc009870 tx=0x7facdc01c530 comp rx=0 tx=0).stop 2024-05-04T21:20:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.327+0000 7faced4eb640 1 -- 172.21.15.52:0/3483538066 shutdown_connections 2024-05-04T21:20:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.327+0000 7faced4eb640 1 --2- 172.21.15.52:0/3483538066 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7face80b31c0 0x7face80ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.327+0000 7faced4eb640 1 --2- 172.21.15.52:0/3483538066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7face8148e00 0x7face8071210 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.327+0000 7faced4eb640 1 -- 172.21.15.52:0/3483538066 >> 172.21.15.52:0/3483538066 conn(0x7face8067070 msgr2=0x7face8067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.327+0000 7faced4eb640 1 -- 172.21.15.52:0/3483538066 shutdown_connections 2024-05-04T21:20:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.327+0000 7faced4eb640 1 -- 172.21.15.52:0/3483538066 wait complete. 2024-05-04T21:20:22.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7faced4eb640 1 Processor -- start 2024-05-04T21:20:22.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7faced4eb640 1 -- start start 2024-05-04T21:20:22.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7faced4eb640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7face80b31c0 0x7face8161e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:22.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7faced4eb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7face8148a20 0x7face8143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:22.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7faced4eb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7face8148e00 0x7face813da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:22.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7faced4eb640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7face806df00 con 0x7face8148a20 2024-05-04T21:20:22.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7faced4eb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7face806dd80 con 0x7face8148e00 2024-05-04T21:20:22.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7faced4eb640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7face8162390 con 0x7face80b31c0 2024-05-04T21:20:22.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7face77fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7face8148e00 0x7face813da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:22.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7face6ffd640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7face80b31c0 0x7face8161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:22.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7face77fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7face8148e00 0x7face813da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:47846/0 (socket says 172.21.15.52:47846) 2024-05-04T21:20:22.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7face77fe640 1 -- 172.21.15.52:0/3906246665 learned_addr learned my addr 172.21.15.52:0/3906246665 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:22.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7face6ffd640 1 -- 172.21.15.52:0/3906246665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7face8148e00 msgr2=0x7face813da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:22.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7face6ffd640 1 --2- 172.21.15.52:0/3906246665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7face8148e00 0x7face813da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7face6ffd640 1 -- 172.21.15.52:0/3906246665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7face8148a20 msgr2=0x7face8143ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:22.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7face6ffd640 1 --2- 172.21.15.52:0/3906246665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7face8148a20 0x7face8143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7face6ffd640 1 -- 172.21.15.52:0/3906246665 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face81422f0 con 0x7face80b31c0 2024-05-04T21:20:22.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7face77fe640 1 --2- 172.21.15.52:0/3906246665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7face8148e00 0x7face813da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T21:20:22.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7face6ffd640 1 --2- 172.21.15.52:0/3906246665 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7face80b31c0 0x7face8161e50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7facd800c3c0 tx=0x7facd800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:22.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7faccf7fe640 1 -- 172.21.15.52:0/3906246665 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd80052d0 con 0x7face80b31c0 2024-05-04T21:20:22.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7faccf7fe640 1 -- 172.21.15.52:0/3906246665 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd8005470 con 0x7face80b31c0 2024-05-04T21:20:22.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.331+0000 7faccf7fe640 1 -- 172.21.15.52:0/3906246665 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8007260 con 0x7face80b31c0 2024-05-04T21:20:22.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.335+0000 7faced4eb640 1 -- 172.21.15.52:0/3906246665 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face81425d0 con 0x7face80b31c0 2024-05-04T21:20:22.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.335+0000 7faced4eb640 1 -- 172.21.15.52:0/3906246665 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7face8142b90 con 0x7face80b31c0 2024-05-04T21:20:22.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.335+0000 7faced4eb640 1 -- 172.21.15.52:0/3906246665 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face8062e50 con 0x7face80b31c0 2024-05-04T21:20:22.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.339+0000 7faccf7fe640 1 -- 172.21.15.52:0/3906246665 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7facd8007400 con 0x7face80b31c0 2024-05-04T21:20:22.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.339+0000 7faccf7fe640 1 --2- 172.21.15.52:0/3906246665 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7facd40678c0 0x7facd4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:22.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.339+0000 7faccf7fe640 1 -- 172.21.15.52:0/3906246665 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7facd80859a0 con 0x7face80b31c0 2024-05-04T21:20:22.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.339+0000 7face67fc640 1 --2- 172.21.15.52:0/3906246665 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7facd40678c0 0x7facd4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:22.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.339+0000 7face67fc640 1 --2- 172.21.15.52:0/3906246665 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7facd40678c0 0x7facd4069d80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7facdc0ae000 tx=0x7facdc004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:22.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.347+0000 7faccf7fe640 1 -- 172.21.15.52:0/3906246665 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7facd8055df0 con 0x7face80b31c0 2024-05-04T21:20:22.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.579+0000 7faced4eb640 1 -- 172.21.15.52:0/3906246665 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7face8075320 con 0x7face80b31c0 2024-05-04T21:20:22.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.635+0000 7faccf7fe640 1 -- 172.21.15.52:0/3906246665 <== mon.2 v2:172.21.15.111:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16681 (secure 0 0 0) 0x7facd805a0f0 con 0x7face80b31c0 2024-05-04T21:20:22.639 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:22.639 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":34,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T12:19:23.639019+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-04T12:19:20.589701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-04T12:19:20.997186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T21:20:22.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.635+0000 7faced4eb640 1 -- 172.21.15.52:0/3906246665 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7facd40678c0 msgr2=0x7facd4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:22.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.635+0000 7faced4eb640 1 --2- 172.21.15.52:0/3906246665 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7facd40678c0 0x7facd4069d80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7facdc0ae000 tx=0x7facdc004740 comp rx=0 tx=0).stop 2024-05-04T21:20:22.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.635+0000 7faced4eb640 1 -- 172.21.15.52:0/3906246665 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7face80b31c0 msgr2=0x7face8161e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:22.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.635+0000 7faced4eb640 1 --2- 172.21.15.52:0/3906246665 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7face80b31c0 0x7face8161e50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7facd800c3c0 tx=0x7facd800c880 comp rx=0 tx=0).stop 2024-05-04T21:20:22.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.635+0000 7faced4eb640 1 -- 172.21.15.52:0/3906246665 shutdown_connections 2024-05-04T21:20:22.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.635+0000 7faced4eb640 1 --2- 172.21.15.52:0/3906246665 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7facd40678c0 0x7facd4069d80 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.635+0000 7faced4eb640 1 --2- 172.21.15.52:0/3906246665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7face8148e00 0x7face813da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.635+0000 7faced4eb640 1 --2- 172.21.15.52:0/3906246665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7face8148a20 0x7face8143ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.635+0000 7faced4eb640 1 --2- 172.21.15.52:0/3906246665 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7face80b31c0 0x7face8161e50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.635+0000 7faced4eb640 1 -- 172.21.15.52:0/3906246665 >> 172.21.15.52:0/3906246665 conn(0x7face8067070 msgr2=0x7face814ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:22.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.635+0000 7faced4eb640 1 -- 172.21.15.52:0/3906246665 shutdown_connections 2024-05-04T21:20:22.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.635+0000 7faced4eb640 1 -- 172.21.15.52:0/3906246665 wait complete. 2024-05-04T21:20:22.669 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-04T21:20:22.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbe00cf640 1 Processor -- start 2024-05-04T21:20:22.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbe00cf640 1 -- start start 2024-05-04T21:20:22.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbe00cf640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbbd813dec0 0x7fbbd8072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:22.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbe00cf640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbd806e600 0x7fbbd813eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:22.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbe00cf640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbd813e630 con 0x7fbbd80733c0 2024-05-04T21:20:22.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbe00cf640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fbbd806ea40 con 0x7fbbd813dec0 2024-05-04T21:20:22.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbe00cf640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd806ec90 con 0x7fbbd806e600 2024-05-04T21:20:22.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbdd66a640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fbbd80733c0 0x7fbbd806def0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:45162/0 (socket says 172.21.15.52:45162) 2024-05-04T21:20:22.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbdd66a640 1 -- 172.21.15.52:0/1401687858 learned_addr learned my addr 172.21.15.52:0/1401687858 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:22.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbde66c640 1 --2- 172.21.15.52:0/1401687858 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbd806e600 0x7fbbd813eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:22.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbdde6b640 1 --2- 172.21.15.52:0/1401687858 >> v2:172.21.15.111:3300/0 conn(0x7fbbd813dec0 0x7fbbd8072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:22.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbdce69640 1 -- 172.21.15.52:0/1401687858 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2419519887 0 0) 0x7fbbd813e630 con 0x7fbbd80733c0 2024-05-04T21:20:22.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbdce69640 1 -- 172.21.15.52:0/1401687858 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbc4003530 con 0x7fbbd80733c0 2024-05-04T21:20:22.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbde66c640 1 -- 172.21.15.52:0/1401687858 >> v2:172.21.15.111:3300/0 conn(0x7fbbd813dec0 msgr2=0x7fbbd8072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:22.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbde66c640 1 --2- 172.21.15.52:0/1401687858 >> v2:172.21.15.111:3300/0 conn(0x7fbbd813dec0 0x7fbbd8072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbde66c640 1 -- 172.21.15.52:0/1401687858 >> v1:172.21.15.52:6789/0 conn(0x7fbbd80733c0 legacy=0x7fbbd806def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:22.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.807+0000 7fbbde66c640 1 -- 172.21.15.52:0/1401687858 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd806ef70 con 0x7fbbd806e600 2024-05-04T21:20:22.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.811+0000 7fbbde66c640 1 --2- 172.21.15.52:0/1401687858 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbd806e600 0x7fbbd813eb10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009d90 tx=0x7fbbd001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d65b8ed7d0933ece server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:22.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.815+0000 7fbbdce69640 1 -- 172.21.15.52:0/1401687858 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00b2070 con 0x7fbbd806e600 2024-05-04T21:20:22.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.815+0000 7fbbdce69640 1 -- 172.21.15.52:0/1401687858 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd00a5920 con 0x7fbbd806e600 2024-05-04T21:20:22.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.815+0000 7fbbdce69640 1 -- 172.21.15.52:0/1401687858 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00a5cf0 con 0x7fbbd806e600 2024-05-04T21:20:22.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.815+0000 7fbbe00cf640 1 -- 172.21.15.52:0/1401687858 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbd806e600 msgr2=0x7fbbd813eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:22.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.815+0000 7fbbe00cf640 1 --2- 172.21.15.52:0/1401687858 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbd806e600 0x7fbbd813eb10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009d90 tx=0x7fbbd001c970 comp rx=0 tx=0).stop 2024-05-04T21:20:22.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.815+0000 7fbbe00cf640 1 -- 172.21.15.52:0/1401687858 shutdown_connections 2024-05-04T21:20:22.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.815+0000 7fbbe00cf640 1 --2- 172.21.15.52:0/1401687858 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbd806e600 0x7fbbd813eb10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.815+0000 7fbbe00cf640 1 --2- 172.21.15.52:0/1401687858 >> v2:172.21.15.111:3300/0 conn(0x7fbbd813dec0 0x7fbbd8072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.815+0000 7fbbe00cf640 1 -- 172.21.15.52:0/1401687858 >> 172.21.15.52:0/1401687858 conn(0x7fbbd8067070 msgr2=0x7fbbd8067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:22.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.815+0000 7fbbe00cf640 1 -- 172.21.15.52:0/1401687858 shutdown_connections 2024-05-04T21:20:22.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.815+0000 7fbbe00cf640 1 -- 172.21.15.52:0/1401687858 wait complete. 2024-05-04T21:20:22.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbe00cf640 1 Processor -- start 2024-05-04T21:20:22.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbe00cf640 1 -- start start 2024-05-04T21:20:22.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbe00cf640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbd806e600 0x7fbbd80cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbe00cf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd80733c0 0x7fbbd8164ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbe00cf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd813dec0 0x7fbbd814daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbe00cf640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd813f9e0 con 0x7fbbd813dec0 2024-05-04T21:20:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbe00cf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd813f860 con 0x7fbbd80733c0 2024-05-04T21:20:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbe00cf640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd813fb60 con 0x7fbbd806e600 2024-05-04T21:20:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbdde6b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbd806e600 0x7fbbd80cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbdde6b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbd806e600 0x7fbbd80cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.52:35348/0 (socket says 172.21.15.52:35348) 2024-05-04T21:20:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbdde6b640 1 -- 172.21.15.52:0/583511528 learned_addr learned my addr 172.21.15.52:0/583511528 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbdd66a640 1 --2- 172.21.15.52:0/583511528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd80733c0 0x7fbbd8164ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbde66c640 1 --2- 172.21.15.52:0/583511528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd813dec0 0x7fbbd814daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbdde6b640 1 -- 172.21.15.52:0/583511528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd80733c0 msgr2=0x7fbbd8164ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbdde6b640 1 --2- 172.21.15.52:0/583511528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd80733c0 0x7fbbd8164ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbdde6b640 1 -- 172.21.15.52:0/583511528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd813dec0 msgr2=0x7fbbd814daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbdde6b640 1 --2- 172.21.15.52:0/583511528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd813dec0 0x7fbbd814daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbdde6b640 1 -- 172.21.15.52:0/583511528 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd81522b0 con 0x7fbbd806e600 2024-05-04T21:20:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbdde6b640 1 --2- 172.21.15.52:0/583511528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbd806e600 0x7fbbd80cf5f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800e870 tx=0x7fbbc800ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbce7fc640 1 -- 172.21.15.52:0/583511528 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8017070 con 0x7fbbd806e600 2024-05-04T21:20:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbce7fc640 1 -- 172.21.15.52:0/583511528 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc800c940 con 0x7fbbd806e600 2024-05-04T21:20:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbe00cf640 1 -- 172.21.15.52:0/583511528 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd814d010 con 0x7fbbd806e600 2024-05-04T21:20:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbce7fc640 1 -- 172.21.15.52:0/583511528 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc800ccf0 con 0x7fbbd806e600 2024-05-04T21:20:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.819+0000 7fbbe00cf640 1 -- 172.21.15.52:0/583511528 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd8165430 con 0x7fbbd806e600 2024-05-04T21:20:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.823+0000 7fbbbbfff640 1 -- 172.21.15.52:0/583511528 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd8062e50 con 0x7fbbd806e600 2024-05-04T21:20:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.823+0000 7fbbce7fc640 1 -- 172.21.15.52:0/583511528 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbbc801a070 con 0x7fbbd806e600 2024-05-04T21:20:22.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.823+0000 7fbbce7fc640 1 --2- 172.21.15.52:0/583511528 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbbbc0678f0 0x7fbbbc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:22.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.823+0000 7fbbce7fc640 1 -- 172.21.15.52:0/583511528 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fbbc8085b00 con 0x7fbbd806e600 2024-05-04T21:20:22.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.823+0000 7fbbdd66a640 1 --2- 172.21.15.52:0/583511528 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbbbc0678f0 0x7fbbbc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.831+0000 7fbbdd66a640 1 --2- 172.21.15.52:0/583511528 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbbbc0678f0 0x7fbbbc069db0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4007fd0 tx=0x7fbbd4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:22.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:22.835+0000 7fbbce7fc640 1 -- 172.21.15.52:0/583511528 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbbc8055f50 con 0x7fbbd806e600 2024-05-04T21:20:23.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.107+0000 7fbbbbfff640 1 -- 172.21.15.52:0/583511528 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbbd8075320 con 0x7fbbd806e600 2024-05-04T21:20:23.180 INFO:tasks.ceph.mon.a.smithi052.stderr:2024-05-04T21:20:23.175+0000 7fc9c3c17640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-04T21:20:23.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.231+0000 7fbbce7fc640 1 -- 172.21.15.52:0/583511528 <== mon.2 v2:172.21.15.111:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7fbbc8085e20 con 0x7fbbd806e600 2024-05-04T21:20:23.235 INFO:teuthology.orchestra.run.smithi052.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-04T21:20:23.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.231+0000 7fbbbbfff640 1 -- 172.21.15.52:0/583511528 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbbbc0678f0 msgr2=0x7fbbbc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.231+0000 7fbbbbfff640 1 --2- 172.21.15.52:0/583511528 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbbbc0678f0 0x7fbbbc069db0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4007fd0 tx=0x7fbbd4009040 comp rx=0 tx=0).stop 2024-05-04T21:20:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.231+0000 7fbbbbfff640 1 -- 172.21.15.52:0/583511528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbd806e600 msgr2=0x7fbbd80cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.231+0000 7fbbbbfff640 1 --2- 172.21.15.52:0/583511528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbd806e600 0x7fbbd80cf5f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800e870 tx=0x7fbbc800ec70 comp rx=0 tx=0).stop 2024-05-04T21:20:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.231+0000 7fbbbbfff640 1 -- 172.21.15.52:0/583511528 shutdown_connections 2024-05-04T21:20:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.231+0000 7fbbbbfff640 1 --2- 172.21.15.52:0/583511528 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbbbc0678f0 0x7fbbbc069db0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.231+0000 7fbbbbfff640 1 --2- 172.21.15.52:0/583511528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd813dec0 0x7fbbd814daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.231+0000 7fbbbbfff640 1 --2- 172.21.15.52:0/583511528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd80733c0 0x7fbbd8164ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.231+0000 7fbbbbfff640 1 --2- 172.21.15.52:0/583511528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbd806e600 0x7fbbd80cf5f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.231+0000 7fbbbbfff640 1 -- 172.21.15.52:0/583511528 >> 172.21.15.52:0/583511528 conn(0x7fbbd8067070 msgr2=0x7fbbd8074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.231+0000 7fbbbbfff640 1 -- 172.21.15.52:0/583511528 shutdown_connections 2024-05-04T21:20:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.231+0000 7fbbbbfff640 1 -- 172.21.15.52:0/583511528 wait complete. 2024-05-04T21:20:23.259 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-04T21:20:23.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faede0e8640 1 Processor -- start 2024-05-04T21:20:23.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faede0e8640 1 -- start start 2024-05-04T21:20:23.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faede0e8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faed815f2b0 0x7faed8162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:23.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faede0e8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7faed80b6110 0x7faed80b6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:23.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faede0e8640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faed80b6a50 con 0x7faed81437b0 2024-05-04T21:20:23.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faede0e8640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7faed8162b80 con 0x7faed80b6110 2024-05-04T21:20:23.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faede0e8640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7faed8141470 con 0x7faed815f2b0 2024-05-04T21:20:23.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faed7fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7faed80b6110 0x7faed80b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:23.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faed7fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7faed80b6110 0x7faed80b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36912/0 (socket says 172.21.15.52:36912) 2024-05-04T21:20:23.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faed7fff640 1 -- 172.21.15.52:0/761363312 learned_addr learned my addr 172.21.15.52:0/761363312 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:23.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faed77fe640 1 --2- 172.21.15.52:0/761363312 >> v2:172.21.15.111:3300/0 conn(0x7faed815f2b0 0x7faed8162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:23.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faed7fff640 1 -- 172.21.15.52:0/761363312 >> v2:172.21.15.111:3300/0 conn(0x7faed815f2b0 msgr2=0x7faed8162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:23.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faed7fff640 1 --2- 172.21.15.52:0/761363312 >> v2:172.21.15.111:3300/0 conn(0x7faed815f2b0 0x7faed8162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:23.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faed7fff640 1 -- 172.21.15.52:0/761363312 >> v1:172.21.15.111:6789/0 conn(0x7faed81437b0 legacy=0x7faed80b9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:23.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.391+0000 7faed7fff640 1 -- 172.21.15.52:0/761363312 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed8143d30 con 0x7faed80b6110 2024-05-04T21:20:23.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faed7fff640 1 --2- 172.21.15.52:0/761363312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faed80b6110 0x7faed80b6510 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7faed80b7070 tx=0x7faec801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3b6cfc8ff470d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:23.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faed67fc640 1 -- 172.21.15.52:0/761363312 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec80a56c0 con 0x7faed80b6110 2024-05-04T21:20:23.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faed67fc640 1 -- 172.21.15.52:0/761363312 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faec80a5860 con 0x7faed80b6110 2024-05-04T21:20:23.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faed67fc640 1 -- 172.21.15.52:0/761363312 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec80a5c30 con 0x7faed80b6110 2024-05-04T21:20:23.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 -- 172.21.15.52:0/761363312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faed80b6110 msgr2=0x7faed80b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:23.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 --2- 172.21.15.52:0/761363312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faed80b6110 0x7faed80b6510 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7faed80b7070 tx=0x7faec801c9f0 comp rx=0 tx=0).stop 2024-05-04T21:20:23.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 -- 172.21.15.52:0/761363312 shutdown_connections 2024-05-04T21:20:23.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 --2- 172.21.15.52:0/761363312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faed80b6110 0x7faed80b6510 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:23.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 --2- 172.21.15.52:0/761363312 >> v2:172.21.15.111:3300/0 conn(0x7faed815f2b0 0x7faed8162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:23.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 -- 172.21.15.52:0/761363312 >> 172.21.15.52:0/761363312 conn(0x7faed80b3510 msgr2=0x7faed80b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:23.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 -- 172.21.15.52:0/761363312 shutdown_connections 2024-05-04T21:20:23.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 -- 172.21.15.52:0/761363312 wait complete. 2024-05-04T21:20:23.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 Processor -- start 2024-05-04T21:20:23.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 -- start start 2024-05-04T21:20:23.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faed80b6110 0x7faed8159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:23.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed81437b0 0x7faed813ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:23.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faed815f2b0 0x7faed813d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:23.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faed81445d0 con 0x7faed815f2b0 2024-05-04T21:20:23.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faed815ea30 con 0x7faed81437b0 2024-05-04T21:20:23.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.395+0000 7faede0e8640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7faed816b000 con 0x7faed80b6110 2024-05-04T21:20:23.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faed7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faed815f2b0 0x7faed813d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:23.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faed7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faed815f2b0 0x7faed813d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36920/0 (socket says 172.21.15.52:36920) 2024-05-04T21:20:23.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faed7fff640 1 -- 172.21.15.52:0/2442989586 learned_addr learned my addr 172.21.15.52:0/2442989586 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:23.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faed77fe640 1 --2- 172.21.15.52:0/2442989586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faed80b6110 0x7faed8159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:23.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faed6ffd640 1 --2- 172.21.15.52:0/2442989586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed81437b0 0x7faed813ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:23.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faed7fff640 1 -- 172.21.15.52:0/2442989586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faed80b6110 msgr2=0x7faed8159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:23.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faed7fff640 1 --2- 172.21.15.52:0/2442989586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faed80b6110 0x7faed8159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:23.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faed7fff640 1 -- 172.21.15.52:0/2442989586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed81437b0 msgr2=0x7faed813ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:23.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faed7fff640 1 --2- 172.21.15.52:0/2442989586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed81437b0 0x7faed813ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:23.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faed7fff640 1 -- 172.21.15.52:0/2442989586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed8159840 con 0x7faed815f2b0 2024-05-04T21:20:23.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faed7fff640 1 --2- 172.21.15.52:0/2442989586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faed815f2b0 0x7faed813d710 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7faec80af040 tx=0x7faec8019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:23.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faecffff640 1 -- 172.21.15.52:0/2442989586 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec80bc020 con 0x7faed815f2b0 2024-05-04T21:20:23.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faecffff640 1 -- 172.21.15.52:0/2442989586 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faec80b8670 con 0x7faed815f2b0 2024-05-04T21:20:23.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faede0e8640 1 -- 172.21.15.52:0/2442989586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed8159b20 con 0x7faed815f2b0 2024-05-04T21:20:23.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faecffff640 1 -- 172.21.15.52:0/2442989586 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec80b8a40 con 0x7faed815f2b0 2024-05-04T21:20:23.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faede0e8640 1 -- 172.21.15.52:0/2442989586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed815a0e0 con 0x7faed815f2b0 2024-05-04T21:20:23.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faede0e8640 1 -- 172.21.15.52:0/2442989586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed8062e50 con 0x7faed815f2b0 2024-05-04T21:20:23.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faecffff640 1 -- 172.21.15.52:0/2442989586 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faec80bf020 con 0x7faed815f2b0 2024-05-04T21:20:23.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faecffff640 1 --2- 172.21.15.52:0/2442989586 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faeb80678c0 0x7faeb8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:23.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faed77fe640 1 --2- 172.21.15.52:0/2442989586 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faeb80678c0 0x7faeb8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:23.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.399+0000 7faed77fe640 1 --2- 172.21.15.52:0/2442989586 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faeb80678c0 0x7faeb8069d80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7faed80d1ab0 tx=0x7faed0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:23.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.407+0000 7faecffff640 1 -- 172.21.15.52:0/2442989586 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7faec80eefe0 con 0x7faed815f2b0 2024-05-04T21:20:23.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.411+0000 7faecffff640 1 -- 172.21.15.52:0/2442989586 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faec80c6030 con 0x7faed815f2b0 2024-05-04T21:20:23.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:23.603+0000 7faede0e8640 1 -- 172.21.15.52:0/2442989586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7faed8168dd0 con 0x7faed815f2b0 2024-05-04T21:20:23.803 INFO:tasks.ceph.mds.c.smithi052.stdout:starting mds.c at 2024-05-04T21:20:24.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.179+0000 7faecffff640 1 -- 172.21.15.52:0/2442989586 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7faec80165e0 con 0x7faed815f2b0 2024-05-04T21:20:24.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.183+0000 7faede0e8640 1 -- 172.21.15.52:0/2442989586 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faeb80678c0 msgr2=0x7faeb8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:24.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.183+0000 7faede0e8640 1 --2- 172.21.15.52:0/2442989586 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faeb80678c0 0x7faeb8069d80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7faed80d1ab0 tx=0x7faed0009190 comp rx=0 tx=0).stop 2024-05-04T21:20:24.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.183+0000 7faede0e8640 1 -- 172.21.15.52:0/2442989586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faed815f2b0 msgr2=0x7faed813d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:24.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.183+0000 7faede0e8640 1 --2- 172.21.15.52:0/2442989586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faed815f2b0 0x7faed813d710 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7faec80af040 tx=0x7faec8019790 comp rx=0 tx=0).stop 2024-05-04T21:20:24.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.183+0000 7faede0e8640 1 -- 172.21.15.52:0/2442989586 shutdown_connections 2024-05-04T21:20:24.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.183+0000 7faede0e8640 1 --2- 172.21.15.52:0/2442989586 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7faeb80678c0 0x7faeb8069d80 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:24.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.183+0000 7faede0e8640 1 --2- 172.21.15.52:0/2442989586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faed815f2b0 0x7faed813d710 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:24.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.183+0000 7faede0e8640 1 --2- 172.21.15.52:0/2442989586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed81437b0 0x7faed813ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:24.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.183+0000 7faede0e8640 1 --2- 172.21.15.52:0/2442989586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faed80b6110 0x7faed8159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:24.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.183+0000 7faede0e8640 1 -- 172.21.15.52:0/2442989586 >> 172.21.15.52:0/2442989586 conn(0x7faed80b3510 msgr2=0x7faed80b9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:24.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.183+0000 7faede0e8640 1 -- 172.21.15.52:0/2442989586 shutdown_connections 2024-05-04T21:20:24.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.183+0000 7faede0e8640 1 -- 172.21.15.52:0/2442989586 wait complete. 2024-05-04T21:20:24.211 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-04T21:20:24.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.339+0000 7f5bba27e640 1 Processor -- start 2024-05-04T21:20:24.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.339+0000 7f5bba27e640 1 -- start start 2024-05-04T21:20:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.339+0000 7f5bba27e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb415be40 0x7f5bb4070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.339+0000 7f5bba27e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5bb4070c40 0x7f5bb40c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.339+0000 7f5bba27e640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bb40cfe00 con 0x7f5bb40c52d0 2024-05-04T21:20:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.339+0000 7f5bba27e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f5bb40710a0 con 0x7f5bb4070c40 2024-05-04T21:20:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.339+0000 7f5bba27e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb40c5a10 con 0x7f5bb415be40 2024-05-04T21:20:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb881b640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f5bb40c52d0 0x7f5bb40c3b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:45178/0 (socket says 172.21.15.52:45178) 2024-05-04T21:20:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb881b640 1 -- 172.21.15.52:0/465877113 learned_addr learned my addr 172.21.15.52:0/465877113 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb37fe640 1 --2- 172.21.15.52:0/465877113 >> v2:172.21.15.111:3300/0 conn(0x7f5bb4070c40 0x7f5bb40c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb3fff640 1 --2- 172.21.15.52:0/465877113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb415be40 0x7f5bb4070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/465877113 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1733243163 0 0) 0x7f5bb40cfe00 con 0x7f5bb40c52d0 2024-05-04T21:20:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/465877113 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b9c003530 con 0x7f5bb40c52d0 2024-05-04T21:20:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/465877113 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2794491206 0 0) 0x7f5b9c003530 con 0x7f5bb40c52d0 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/465877113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb415be40 msgr2=0x7f5bb4070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/465877113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb415be40 0x7f5bb4070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/465877113 >> v2:172.21.15.111:3300/0 conn(0x7f5bb4070c40 msgr2=0x7f5bb40c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/465877113 >> v2:172.21.15.111:3300/0 conn(0x7f5bb4070c40 0x7f5bb40c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/465877113 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb40d00c0 con 0x7f5bb40c52d0 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/465877113 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f5bac001680 con 0x7f5bb40c52d0 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/465877113 >> v1:172.21.15.52:6789/0 conn(0x7f5bb40c52d0 legacy=0x7f5bb40c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/465877113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b9c0039c0 0x7f5b9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/465877113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb415be40 0x7f5b9c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/465877113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c004d50 0x7f5b9c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/465877113 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb40c5a10 con 0x7f5b9c004d50 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/465877113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c00b9d0 con 0x7f5b9c0039c0 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/465877113 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5b9c00bb50 con 0x7f5bb415be40 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb881b640 1 --2- 172.21.15.52:0/465877113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c004d50 0x7f5b9c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb3fff640 1 --2- 172.21.15.52:0/465877113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b9c0039c0 0x7f5b9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb37fe640 1 --2- 172.21.15.52:0/465877113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb415be40 0x7f5b9c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb881b640 1 -- 172.21.15.52:0/465877113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb415be40 msgr2=0x7f5b9c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb881b640 1 --2- 172.21.15.52:0/465877113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb415be40 0x7f5b9c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb881b640 1 -- 172.21.15.52:0/465877113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b9c0039c0 msgr2=0x7f5b9c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb881b640 1 --2- 172.21.15.52:0/465877113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b9c0039c0 0x7f5b9c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb881b640 1 -- 172.21.15.52:0/465877113 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c003530 con 0x7f5b9c004d50 2024-05-04T21:20:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb3fff640 1 --2- 172.21.15.52:0/465877113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b9c0039c0 0x7f5b9c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T21:20:24.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb881b640 1 --2- 172.21.15.52:0/465877113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c004d50 0x7f5b9c0051d0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00c210 tx=0x7f5bac01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10e46119bfe8c13 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:24.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/465877113 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac007d60 con 0x7f5b9c004d50 2024-05-04T21:20:24.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/465877113 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bac0a9020 con 0x7f5b9c004d50 2024-05-04T21:20:24.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.343+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/465877113 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0a5be0 con 0x7f5b9c004d50 2024-05-04T21:20:24.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 -- 172.21.15.52:0/465877113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c004d50 msgr2=0x7f5b9c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:24.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 --2- 172.21.15.52:0/465877113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c004d50 0x7f5b9c0051d0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00c210 tx=0x7f5bac01cca0 comp rx=0 tx=0).stop 2024-05-04T21:20:24.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb881b640 1 -- 172.21.15.52:0/465877113 reap_dead start 2024-05-04T21:20:24.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 -- 172.21.15.52:0/465877113 shutdown_connections 2024-05-04T21:20:24.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 -- 172.21.15.52:0/465877113 >> 172.21.15.52:0/465877113 conn(0x7f5bb406c450 msgr2=0x7f5bb4067070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:24.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 -- 172.21.15.52:0/465877113 shutdown_connections 2024-05-04T21:20:24.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 -- 172.21.15.52:0/465877113 wait complete. 2024-05-04T21:20:24.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 Processor -- start 2024-05-04T21:20:24.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 -- start start 2024-05-04T21:20:24.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb4140f20 0x7f5bb40c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:24.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bb4141300 0x7f5bb41447f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:24.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb40c9f30 0x7f5bb4144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:24.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb40c5730 con 0x7f5bb4140f20 2024-05-04T21:20:24.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb40d0680 con 0x7f5bb4141300 2024-05-04T21:20:24.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb40d0ba0 con 0x7f5bb40c9f30 2024-05-04T21:20:24.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb4140f20 0x7f5bb40c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:24.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb4140f20 0x7f5bb40c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36958/0 (socket says 172.21.15.52:36958) 2024-05-04T21:20:24.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb3fff640 1 -- 172.21.15.52:0/1107250474 learned_addr learned my addr 172.21.15.52:0/1107250474 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:24.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb37fe640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bb4141300 0x7f5bb41447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:24.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb881b640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb40c9f30 0x7f5bb4144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:24.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb3fff640 1 -- 172.21.15.52:0/1107250474 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb40c9f30 msgr2=0x7f5bb4144d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:24.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb3fff640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb40c9f30 0x7f5bb4144d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:24.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb3fff640 1 -- 172.21.15.52:0/1107250474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bb4141300 msgr2=0x7f5bb41447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:24.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb3fff640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bb4141300 0x7f5bb41447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:24.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb3fff640 1 -- 172.21.15.52:0/1107250474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb4143a30 con 0x7f5bb4140f20 2024-05-04T21:20:24.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb881b640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb40c9f30 0x7f5bb4144d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:24.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb37fe640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bb4141300 0x7f5bb41447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:24.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb3fff640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb4140f20 0x7f5bb40c99f0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4007f30 tx=0x7f5ba4010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:24.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb0ff9640 1 -- 172.21.15.52:0/1107250474 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba401a030 con 0x7f5bb4140f20 2024-05-04T21:20:24.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb0ff9640 1 -- 172.21.15.52:0/1107250474 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba40039a0 con 0x7f5bb4140f20 2024-05-04T21:20:24.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 -- 172.21.15.52:0/1107250474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb4143d10 con 0x7f5bb4140f20 2024-05-04T21:20:24.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bb0ff9640 1 -- 172.21.15.52:0/1107250474 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4003d70 con 0x7f5bb4140f20 2024-05-04T21:20:24.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 -- 172.21.15.52:0/1107250474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb41442d0 con 0x7f5bb4140f20 2024-05-04T21:20:24.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.347+0000 7f5bba27e640 1 -- 172.21.15.52:0/1107250474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb4062e50 con 0x7f5bb4140f20 2024-05-04T21:20:24.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.351+0000 7f5bb0ff9640 1 -- 172.21.15.52:0/1107250474 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5ba401d070 con 0x7f5bb4140f20 2024-05-04T21:20:24.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.351+0000 7f5bb0ff9640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5ba00678f0 0x7f5ba0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:24.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.351+0000 7f5bb0ff9640 1 -- 172.21.15.52:0/1107250474 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f5ba4086cf0 con 0x7f5bb4140f20 2024-05-04T21:20:24.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.351+0000 7f5bb37fe640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5ba00678f0 0x7f5ba0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:24.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.351+0000 7f5bb37fe640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5ba00678f0 0x7f5ba0069db0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4145510 tx=0x7f5ba8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:24.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.355+0000 7f5bb0ff9640 1 -- 172.21.15.52:0/1107250474 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5ba40577c0 con 0x7f5bb4140f20 2024-05-04T21:20:24.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:24.511+0000 7f5bba27e640 1 -- 172.21.15.52:0/1107250474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5bb406e9c0 con 0x7f5bb4140f20 2024-05-04T21:20:25.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.195+0000 7f5bb0ff9640 1 -- 172.21.15.52:0/1107250474 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) ==== 181+0+0 (secure 0 0 0) 0x7f5ba40570c0 con 0x7f5bb4140f20 2024-05-04T21:20:25.199 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' removed 2024-05-04T21:20:25.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.195+0000 7f5bba27e640 1 -- 172.21.15.52:0/1107250474 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5ba00678f0 msgr2=0x7f5ba0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:25.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.195+0000 7f5bba27e640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5ba00678f0 0x7f5ba0069db0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4145510 tx=0x7f5ba8008040 comp rx=0 tx=0).stop 2024-05-04T21:20:25.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.195+0000 7f5bba27e640 1 -- 172.21.15.52:0/1107250474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb4140f20 msgr2=0x7f5bb40c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:25.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.195+0000 7f5bba27e640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb4140f20 0x7f5bb40c99f0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4007f30 tx=0x7f5ba4010650 comp rx=0 tx=0).stop 2024-05-04T21:20:25.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.195+0000 7f5bba27e640 1 -- 172.21.15.52:0/1107250474 shutdown_connections 2024-05-04T21:20:25.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.195+0000 7f5bba27e640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5ba00678f0 0x7f5ba0069db0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:25.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.195+0000 7f5bba27e640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb40c9f30 0x7f5bb4144d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:25.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.195+0000 7f5bba27e640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bb4141300 0x7f5bb41447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:25.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.195+0000 7f5bba27e640 1 --2- 172.21.15.52:0/1107250474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb4140f20 0x7f5bb40c99f0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:25.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.195+0000 7f5bba27e640 1 -- 172.21.15.52:0/1107250474 >> 172.21.15.52:0/1107250474 conn(0x7f5bb406c450 msgr2=0x7f5bb4072510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:25.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.199+0000 7f5bba27e640 1 -- 172.21.15.52:0/1107250474 shutdown_connections 2024-05-04T21:20:25.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.199+0000 7f5bba27e640 1 -- 172.21.15.52:0/1107250474 wait complete. 2024-05-04T21:20:25.223 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-04T21:20:25.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13d19db640 1 Processor -- start 2024-05-04T21:20:25.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13d19db640 1 -- start start 2024-05-04T21:20:25.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13d19db640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13cc1415c0 0x7f13cc0725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:25.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13d19db640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f13cc0b4780 0x7f13cc1419a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:25.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13d19db640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13cc072ba0 con 0x7f13cc141ee0 2024-05-04T21:20:25.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13d19db640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f13cc15df00 con 0x7f13cc0b4780 2024-05-04T21:20:25.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13d19db640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f13cc072e80 con 0x7f13cc1415c0 2024-05-04T21:20:25.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13ca7fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f13cc0b4780 0x7f13cc1419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:25.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13ca7fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f13cc0b4780 0x7f13cc1419a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36968/0 (socket says 172.21.15.52:36968) 2024-05-04T21:20:25.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13ca7fc640 1 -- 172.21.15.52:0/2773539551 learned_addr learned my addr 172.21.15.52:0/2773539551 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:25.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13caffd640 1 --2- 172.21.15.52:0/2773539551 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13cc1415c0 0x7f13cc0725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:25.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13ca7fc640 1 -- 172.21.15.52:0/2773539551 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13cc1415c0 msgr2=0x7f13cc0725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:25.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13ca7fc640 1 --2- 172.21.15.52:0/2773539551 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13cc1415c0 0x7f13cc0725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:25.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13ca7fc640 1 -- 172.21.15.52:0/2773539551 >> v1:172.21.15.111:6789/0 conn(0x7f13cc141ee0 legacy=0x7f13cc15c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:25.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.359+0000 7f13ca7fc640 1 -- 172.21.15.52:0/2773539551 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc15d8b0 con 0x7f13cc0b4780 2024-05-04T21:20:25.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13ca7fc640 1 --2- 172.21.15.52:0/2773539551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13cc0b4780 0x7f13cc1419a0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009870 tx=0x7f13c001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9708be0922011fb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:25.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13c9ffb640 1 -- 172.21.15.52:0/2773539551 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c00a7070 con 0x7f13cc0b4780 2024-05-04T21:20:25.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13c9ffb640 1 -- 172.21.15.52:0/2773539551 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c00b0070 con 0x7f13cc0b4780 2024-05-04T21:20:25.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13c9ffb640 1 -- 172.21.15.52:0/2773539551 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c00ab890 con 0x7f13cc0b4780 2024-05-04T21:20:25.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 -- 172.21.15.52:0/2773539551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13cc0b4780 msgr2=0x7f13cc1419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:25.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 --2- 172.21.15.52:0/2773539551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13cc0b4780 0x7f13cc1419a0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009870 tx=0x7f13c001c560 comp rx=0 tx=0).stop 2024-05-04T21:20:25.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 -- 172.21.15.52:0/2773539551 shutdown_connections 2024-05-04T21:20:25.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 --2- 172.21.15.52:0/2773539551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13cc0b4780 0x7f13cc1419a0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:25.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 --2- 172.21.15.52:0/2773539551 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13cc1415c0 0x7f13cc0725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:25.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 -- 172.21.15.52:0/2773539551 >> 172.21.15.52:0/2773539551 conn(0x7f13cc070690 msgr2=0x7f13cc070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:25.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 -- 172.21.15.52:0/2773539551 shutdown_connections 2024-05-04T21:20:25.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 -- 172.21.15.52:0/2773539551 wait complete. 2024-05-04T21:20:25.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 Processor -- start 2024-05-04T21:20:25.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 -- start start 2024-05-04T21:20:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13cc0b4780 0x7f13cc140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13cc1415c0 0x7f13cc0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13cc141ee0 0x7f13cc0b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc169020 con 0x7f13cc0b4780 2024-05-04T21:20:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc168ea0 con 0x7f13cc1415c0 2024-05-04T21:20:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13d19db640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f13cc1691a0 con 0x7f13cc141ee0 2024-05-04T21:20:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13caffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13cc0b4780 0x7f13cc140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13caffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13cc0b4780 0x7f13cc140760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36978/0 (socket says 172.21.15.52:36978) 2024-05-04T21:20:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13caffd640 1 -- 172.21.15.52:0/3313163863 learned_addr learned my addr 172.21.15.52:0/3313163863 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.363+0000 7f13ca7fc640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13cc1415c0 0x7f13cc0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13caffd640 1 -- 172.21.15.52:0/3313163863 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13cc141ee0 msgr2=0x7f13cc0b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13cb7fe640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13cc141ee0 0x7f13cc0b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13caffd640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13cc141ee0 0x7f13cc0b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13caffd640 1 -- 172.21.15.52:0/3313163863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13cc1415c0 msgr2=0x7f13cc0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13caffd640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13cc1415c0 0x7f13cc0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13caffd640 1 -- 172.21.15.52:0/3313163863 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc0cfc30 con 0x7f13cc0b4780 2024-05-04T21:20:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13ca7fc640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13cc1415c0 0x7f13cc0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13cb7fe640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13cc141ee0 0x7f13cc0b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13caffd640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13cc0b4780 0x7f13cc140760 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00d800 tx=0x7f13bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13b3fff640 1 -- 172.21.15.52:0/3313163863 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc00b6c0 con 0x7f13cc0b4780 2024-05-04T21:20:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13b3fff640 1 -- 172.21.15.52:0/3313163863 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13bc019070 con 0x7f13cc0b4780 2024-05-04T21:20:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13b3fff640 1 -- 172.21.15.52:0/3313163863 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc00ba60 con 0x7f13cc0b4780 2024-05-04T21:20:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13d19db640 1 -- 172.21.15.52:0/3313163863 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13cc1540b0 con 0x7f13cc0b4780 2024-05-04T21:20:25.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13d19db640 1 -- 172.21.15.52:0/3313163863 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13cc154670 con 0x7f13cc0b4780 2024-05-04T21:20:25.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13d19db640 1 -- 172.21.15.52:0/3313163863 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13cc062e50 con 0x7f13cc0b4780 2024-05-04T21:20:25.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13b3fff640 1 -- 172.21.15.52:0/3313163863 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f13bc00bc00 con 0x7f13cc0b4780 2024-05-04T21:20:25.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13b3fff640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13b80678c0 0x7f13b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:25.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13b3fff640 1 -- 172.21.15.52:0/3313163863 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5357+0+0 (secure 0 0 0) 0x7f13bc085f60 con 0x7f13cc0b4780 2024-05-04T21:20:25.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13ca7fc640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13b80678c0 0x7f13b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:25.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.367+0000 7f13ca7fc640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13b80678c0 0x7f13b8069d80 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f13c00af000 tx=0x7f13c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:25.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.375+0000 7f13b3fff640 1 -- 172.21.15.52:0/3313163863 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f13bc056c90 con 0x7f13cc0b4780 2024-05-04T21:20:25.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:25.539+0000 7f13d19db640 1 -- 172.21.15.52:0/3313163863 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f13cc075320 con 0x7f13cc0b4780 2024-05-04T21:20:26.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.203+0000 7f13b3fff640 1 -- 172.21.15.52:0/3313163863 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) ==== 169+0+0 (secure 0 0 0) 0x7f13bc056590 con 0x7f13cc0b4780 2024-05-04T21:20:26.208 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' removed 2024-05-04T21:20:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.211+0000 7f13d19db640 1 -- 172.21.15.52:0/3313163863 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13b80678c0 msgr2=0x7f13b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.211+0000 7f13d19db640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13b80678c0 0x7f13b8069d80 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f13c00af000 tx=0x7f13c0008040 comp rx=0 tx=0).stop 2024-05-04T21:20:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.211+0000 7f13d19db640 1 -- 172.21.15.52:0/3313163863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13cc0b4780 msgr2=0x7f13cc140760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.211+0000 7f13d19db640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13cc0b4780 0x7f13cc140760 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00d800 tx=0x7f13bc00dcc0 comp rx=0 tx=0).stop 2024-05-04T21:20:26.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.219+0000 7f13d19db640 1 -- 172.21.15.52:0/3313163863 shutdown_connections 2024-05-04T21:20:26.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.219+0000 7f13d19db640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13b80678c0 0x7f13b8069d80 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:26.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.219+0000 7f13d19db640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13cc141ee0 0x7f13cc0b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:26.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.219+0000 7f13d19db640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13cc1415c0 0x7f13cc0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:26.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.219+0000 7f13d19db640 1 --2- 172.21.15.52:0/3313163863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13cc0b4780 0x7f13cc140760 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:26.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.219+0000 7f13d19db640 1 -- 172.21.15.52:0/3313163863 >> 172.21.15.52:0/3313163863 conn(0x7f13cc070690 msgr2=0x7f13cc157740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:26.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.219+0000 7f13d19db640 1 -- 172.21.15.52:0/3313163863 shutdown_connections 2024-05-04T21:20:26.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.219+0000 7f13d19db640 1 -- 172.21.15.52:0/3313163863 wait complete. 2024-05-04T21:20:26.259 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-04T21:20:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe415d01640 1 Processor -- start 2024-05-04T21:20:26.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe415d01640 1 -- start start 2024-05-04T21:20:26.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe415d01640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe410152070 0x7fe4100cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:26.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe415d01640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4100d0950 0x7fe4100a9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:26.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe415d01640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe41015b420 0x7fe4100a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:26.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe415d01640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fe4100cf8b0 con 0x7fe41015b420 2024-05-04T21:20:26.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe415d01640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fe4100cfb90 con 0x7fe4100d0950 2024-05-04T21:20:26.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe415d01640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe4100cfe70 con 0x7fe410152070 2024-05-04T21:20:26.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe40ffff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe41015b420 0x7fe4100a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:26.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe40ffff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe41015b420 0x7fe4100a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59670/0 (socket says 172.21.15.52:59670) 2024-05-04T21:20:26.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe40ffff640 1 -- 172.21.15.52:0/1028834950 learned_addr learned my addr 172.21.15.52:0/1028834950 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:26.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe40f7fe640 1 --2- 172.21.15.52:0/1028834950 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe410152070 0x7fe4100cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:26.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe40effd640 1 --2- 172.21.15.52:0/1028834950 >> v2:172.21.15.111:3300/0 conn(0x7fe4100d0950 0x7fe4100a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:26.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe40ffff640 1 -- 172.21.15.52:0/1028834950 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe410152070 msgr2=0x7fe4100cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:26.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe40ffff640 1 --2- 172.21.15.52:0/1028834950 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe410152070 0x7fe4100cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:26.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe40ffff640 1 -- 172.21.15.52:0/1028834950 >> v2:172.21.15.111:3300/0 conn(0x7fe4100d0950 msgr2=0x7fe4100a9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:26.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe40ffff640 1 --2- 172.21.15.52:0/1028834950 >> v2:172.21.15.111:3300/0 conn(0x7fe4100d0950 0x7fe4100a9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:26.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe40ffff640 1 -- 172.21.15.52:0/1028834950 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4100d0150 con 0x7fe41015b420 2024-05-04T21:20:26.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.471+0000 7fe40effd640 1 --2- 172.21.15.52:0/1028834950 >> v2:172.21.15.111:3300/0 conn(0x7fe4100d0950 0x7fe4100a9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T21:20:26.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.475+0000 7fe40ffff640 1 --2- 172.21.15.52:0/1028834950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe41015b420 0x7fe4100a9c60 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fe41006b270 tx=0x7fe40801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44652bf78afa625 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:26.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.475+0000 7fe40e7fc640 1 -- 172.21.15.52:0/1028834950 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4080a5720 con 0x7fe41015b420 2024-05-04T21:20:26.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.475+0000 7fe40e7fc640 1 -- 172.21.15.52:0/1028834950 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4080a58c0 con 0x7fe41015b420 2024-05-04T21:20:26.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.475+0000 7fe40e7fc640 1 -- 172.21.15.52:0/1028834950 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4080a5c90 con 0x7fe41015b420 2024-05-04T21:20:26.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.475+0000 7fe415d01640 1 -- 172.21.15.52:0/1028834950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe41015b420 msgr2=0x7fe4100a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:26.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.475+0000 7fe415d01640 1 --2- 172.21.15.52:0/1028834950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe41015b420 0x7fe4100a9c60 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fe41006b270 tx=0x7fe40801c9f0 comp rx=0 tx=0).stop 2024-05-04T21:20:26.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.475+0000 7fe415d01640 1 -- 172.21.15.52:0/1028834950 shutdown_connections 2024-05-04T21:20:26.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.475+0000 7fe415d01640 1 --2- 172.21.15.52:0/1028834950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe41015b420 0x7fe4100a9c60 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:26.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.475+0000 7fe415d01640 1 --2- 172.21.15.52:0/1028834950 >> v2:172.21.15.111:3300/0 conn(0x7fe4100d0950 0x7fe4100a9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:26.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.475+0000 7fe415d01640 1 --2- 172.21.15.52:0/1028834950 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe410152070 0x7fe4100cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:26.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.475+0000 7fe415d01640 1 -- 172.21.15.52:0/1028834950 >> 172.21.15.52:0/1028834950 conn(0x7fe4100b24b0 msgr2=0x7fe4100b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:26.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.475+0000 7fe415d01640 1 -- 172.21.15.52:0/1028834950 shutdown_connections 2024-05-04T21:20:26.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.475+0000 7fe415d01640 1 -- 172.21.15.52:0/1028834950 wait complete. 2024-05-04T21:20:26.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe415d01640 1 Processor -- start 2024-05-04T21:20:26.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe415d01640 1 -- start start 2024-05-04T21:20:26.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe415d01640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4100d0950 0x7fe4101443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:26.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe415d01640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe410152070 0x7fe41014a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:26.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe415d01640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe41015b420 0x7fe41014fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:26.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe415d01640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe41015f590 con 0x7fe410152070 2024-05-04T21:20:26.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe415d01640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe41015f410 con 0x7fe4100d0950 2024-05-04T21:20:26.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe415d01640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe41015f710 con 0x7fe41015b420 2024-05-04T21:20:26.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe40f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4100d0950 0x7fe4101443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:26.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe40effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe410152070 0x7fe41014a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:26.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe40f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4100d0950 0x7fe4101443d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:51610/0 (socket says 172.21.15.52:51610) 2024-05-04T21:20:26.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe40f7fe640 1 -- 172.21.15.52:0/2940013040 learned_addr learned my addr 172.21.15.52:0/2940013040 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:26.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe40ffff640 1 --2- 172.21.15.52:0/2940013040 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe41015b420 0x7fe41014fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:26.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe40f7fe640 1 -- 172.21.15.52:0/2940013040 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe41015b420 msgr2=0x7fe41014fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:26.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe40f7fe640 1 --2- 172.21.15.52:0/2940013040 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe41015b420 0x7fe41014fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:26.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe40f7fe640 1 -- 172.21.15.52:0/2940013040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe410152070 msgr2=0x7fe41014a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:26.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe40f7fe640 1 --2- 172.21.15.52:0/2940013040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe410152070 0x7fe41014a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:26.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe40f7fe640 1 -- 172.21.15.52:0/2940013040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4101497b0 con 0x7fe4100d0950 2024-05-04T21:20:26.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.479+0000 7fe40effd640 1 --2- 172.21.15.52:0/2940013040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe410152070 0x7fe41014a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T21:20:26.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.483+0000 7fe40f7fe640 1 --2- 172.21.15.52:0/2940013040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4100d0950 0x7fe4101443d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe40000e8a0 tx=0x7fe40000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:26.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.483+0000 7fe3f7fff640 1 -- 172.21.15.52:0/2940013040 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400017070 con 0x7fe4100d0950 2024-05-04T21:20:26.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.483+0000 7fe3f7fff640 1 -- 172.21.15.52:0/2940013040 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe40000c940 con 0x7fe4100d0950 2024-05-04T21:20:26.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.483+0000 7fe415d01640 1 -- 172.21.15.52:0/2940013040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe410149a90 con 0x7fe4100d0950 2024-05-04T21:20:26.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.483+0000 7fe3f7fff640 1 -- 172.21.15.52:0/2940013040 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40000ccf0 con 0x7fe4100d0950 2024-05-04T21:20:26.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.483+0000 7fe415d01640 1 -- 172.21.15.52:0/2940013040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41014a050 con 0x7fe4100d0950 2024-05-04T21:20:26.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.483+0000 7fe415d01640 1 -- 172.21.15.52:0/2940013040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe410062e50 con 0x7fe4100d0950 2024-05-04T21:20:26.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.487+0000 7fe3f7fff640 1 -- 172.21.15.52:0/2940013040 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe40001a070 con 0x7fe4100d0950 2024-05-04T21:20:26.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.487+0000 7fe3f7fff640 1 --2- 172.21.15.52:0/2940013040 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe3fc067910 0x7fe3fc069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:26.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.487+0000 7fe3f7fff640 1 -- 172.21.15.52:0/2940013040 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fe400085b10 con 0x7fe4100d0950 2024-05-04T21:20:26.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.487+0000 7fe40effd640 1 --2- 172.21.15.52:0/2940013040 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe3fc067910 0x7fe3fc069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:26.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.487+0000 7fe40effd640 1 --2- 172.21.15.52:0/2940013040 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe3fc067910 0x7fe3fc069dd0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe410144fa0 tx=0x7fe404008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:26.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.495+0000 7fe3f7fff640 1 -- 172.21.15.52:0/2940013040 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe400056370 con 0x7fe4100d0950 2024-05-04T21:20:26.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:26.915+0000 7fe415d01640 1 -- 172.21.15.52:0/2940013040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe410075320 con 0x7fe4100d0950 2024-05-04T21:20:27.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.011+0000 7fe3f7fff640 1 -- 172.21.15.52:0/2940013040 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fe40005a670 con 0x7fe4100d0950 2024-05-04T21:20:27.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.031+0000 7fe415d01640 1 -- 172.21.15.52:0/2940013040 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe3fc067910 msgr2=0x7fe3fc069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:27.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.031+0000 7fe415d01640 1 --2- 172.21.15.52:0/2940013040 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe3fc067910 0x7fe3fc069dd0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe410144fa0 tx=0x7fe404008040 comp rx=0 tx=0).stop 2024-05-04T21:20:27.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.031+0000 7fe415d01640 1 -- 172.21.15.52:0/2940013040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4100d0950 msgr2=0x7fe4101443d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:27.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.031+0000 7fe415d01640 1 --2- 172.21.15.52:0/2940013040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4100d0950 0x7fe4101443d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe40000e8a0 tx=0x7fe40000eca0 comp rx=0 tx=0).stop 2024-05-04T21:20:27.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.031+0000 7fe415d01640 1 -- 172.21.15.52:0/2940013040 shutdown_connections 2024-05-04T21:20:27.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.031+0000 7fe415d01640 1 --2- 172.21.15.52:0/2940013040 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe3fc067910 0x7fe3fc069dd0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:27.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.031+0000 7fe415d01640 1 --2- 172.21.15.52:0/2940013040 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe41015b420 0x7fe41014fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:27.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.031+0000 7fe415d01640 1 --2- 172.21.15.52:0/2940013040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe410152070 0x7fe41014a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:27.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.031+0000 7fe415d01640 1 --2- 172.21.15.52:0/2940013040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4100d0950 0x7fe4101443d0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:27.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.031+0000 7fe415d01640 1 -- 172.21.15.52:0/2940013040 >> 172.21.15.52:0/2940013040 conn(0x7fe4100b24b0 msgr2=0x7fe410074a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:27.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.031+0000 7fe415d01640 1 -- 172.21.15.52:0/2940013040 shutdown_connections 2024-05-04T21:20:27.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.035+0000 7fe415d01640 1 -- 172.21.15.52:0/2940013040 wait complete. 2024-05-04T21:20:27.132 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2024-05-04T21:20:27.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.299+0000 7fce2486a640 1 Processor -- start 2024-05-04T21:20:27.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.299+0000 7fce2486a640 1 -- start start 2024-05-04T21:20:27.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.299+0000 7fce2486a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fce20144590 0x7fce200b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:27.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.299+0000 7fce2486a640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce200b6a20 0x7fce20141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:27.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.299+0000 7fce2486a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce2015eac0 con 0x7fce201441b0 2024-05-04T21:20:27.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.299+0000 7fce2486a640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fce200b6e80 con 0x7fce20144590 2024-05-04T21:20:27.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.299+0000 7fce2486a640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fce2015f6d0 con 0x7fce200b6a20 2024-05-04T21:20:27.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce17fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fce20144590 0x7fce200b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:27.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce17fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fce20144590 0x7fce200b64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59692/0 (socket says 172.21.15.52:59692) 2024-05-04T21:20:27.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce17fff640 1 -- 172.21.15.52:0/3503644184 learned_addr learned my addr 172.21.15.52:0/3503644184 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:27.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce1ed9d640 1 --2- 172.21.15.52:0/3503644184 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce200b6a20 0x7fce20141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:27.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce17fff640 1 -- 172.21.15.52:0/3503644184 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce200b6a20 msgr2=0x7fce20141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:27.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce17fff640 1 --2- 172.21.15.52:0/3503644184 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce200b6a20 0x7fce20141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:27.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce17fff640 1 -- 172.21.15.52:0/3503644184 >> v1:172.21.15.111:6789/0 conn(0x7fce201441b0 legacy=0x7fce201627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:27.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce17fff640 1 -- 172.21.15.52:0/3503644184 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2015eda0 con 0x7fce20144590 2024-05-04T21:20:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce17fff640 1 --2- 172.21.15.52:0/3503644184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce20144590 0x7fce200b64e0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fce10009870 tx=0x7fce1001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c99a47a6396a716 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce1dd9b640 1 -- 172.21.15.52:0/3503644184 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce100aa610 con 0x7fce20144590 2024-05-04T21:20:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce1dd9b640 1 -- 172.21.15.52:0/3503644184 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce100af070 con 0x7fce20144590 2024-05-04T21:20:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce1dd9b640 1 -- 172.21.15.52:0/3503644184 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce100aa9d0 con 0x7fce20144590 2024-05-04T21:20:27.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce2486a640 1 -- 172.21.15.52:0/3503644184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce20144590 msgr2=0x7fce200b64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:27.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce2486a640 1 --2- 172.21.15.52:0/3503644184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce20144590 0x7fce200b64e0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fce10009870 tx=0x7fce1001c4f0 comp rx=0 tx=0).stop 2024-05-04T21:20:27.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce2486a640 1 -- 172.21.15.52:0/3503644184 shutdown_connections 2024-05-04T21:20:27.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce2486a640 1 --2- 172.21.15.52:0/3503644184 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce200b6a20 0x7fce20141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:27.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce2486a640 1 --2- 172.21.15.52:0/3503644184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce20144590 0x7fce200b64e0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:27.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce2486a640 1 -- 172.21.15.52:0/3503644184 >> 172.21.15.52:0/3503644184 conn(0x7fce200b52d0 msgr2=0x7fce200b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:27.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce2486a640 1 -- 172.21.15.52:0/3503644184 shutdown_connections 2024-05-04T21:20:27.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.303+0000 7fce2486a640 1 -- 172.21.15.52:0/3503644184 wait complete. 2024-05-04T21:20:27.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce2486a640 1 Processor -- start 2024-05-04T21:20:27.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce2486a640 1 -- start start 2024-05-04T21:20:27.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce2486a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce200b6a20 0x7fce201514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:27.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce2486a640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce201441b0 0x7fce201519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:27.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce2486a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce20144590 0x7fce20151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:27.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce2486a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fce20143c10 con 0x7fce20144590 2024-05-04T21:20:27.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce2486a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fce20143a90 con 0x7fce200b6a20 2024-05-04T21:20:27.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce2486a640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fce20142800 con 0x7fce201441b0 2024-05-04T21:20:27.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce1ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce20144590 0x7fce20151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:27.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce1ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce20144590 0x7fce20151f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59706/0 (socket says 172.21.15.52:59706) 2024-05-04T21:20:27.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce1ed9d640 1 -- 172.21.15.52:0/3851265700 learned_addr learned my addr 172.21.15.52:0/3851265700 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:27.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce17fff640 1 --2- 172.21.15.52:0/3851265700 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce201441b0 0x7fce201519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:27.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce1ed9d640 1 -- 172.21.15.52:0/3851265700 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce201441b0 msgr2=0x7fce201519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:27.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce1ed9d640 1 --2- 172.21.15.52:0/3851265700 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce201441b0 0x7fce201519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:27.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce1ed9d640 1 -- 172.21.15.52:0/3851265700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce200b6a20 msgr2=0x7fce201514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:27.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce1ed9d640 1 --2- 172.21.15.52:0/3851265700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce200b6a20 0x7fce201514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:27.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce1ed9d640 1 -- 172.21.15.52:0/3851265700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce201561d0 con 0x7fce20144590 2024-05-04T21:20:27.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce1ed9d640 1 --2- 172.21.15.52:0/3851265700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce20144590 0x7fce20151f20 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fce18007e70 tx=0x7fce1800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:27.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce16ffd640 1 -- 172.21.15.52:0/3851265700 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18014620 con 0x7fce20144590 2024-05-04T21:20:27.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce16ffd640 1 -- 172.21.15.52:0/3851265700 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce18019070 con 0x7fce20144590 2024-05-04T21:20:27.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce16ffd640 1 -- 172.21.15.52:0/3851265700 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18014a20 con 0x7fce20144590 2024-05-04T21:20:27.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce2486a640 1 -- 172.21.15.52:0/3851265700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce201564b0 con 0x7fce20144590 2024-05-04T21:20:27.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce2486a640 1 -- 172.21.15.52:0/3851265700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2013a250 con 0x7fce20144590 2024-05-04T21:20:27.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce2486a640 1 -- 172.21.15.52:0/3851265700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce201569d0 con 0x7fce20144590 2024-05-04T21:20:27.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce16ffd640 1 -- 172.21.15.52:0/3851265700 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fce18014bc0 con 0x7fce20144590 2024-05-04T21:20:27.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce16ffd640 1 --2- 172.21.15.52:0/3851265700 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fce040678c0 0x7fce04069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:27.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce16ffd640 1 -- 172.21.15.52:0/3851265700 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fce18086200 con 0x7fce20144590 2024-05-04T21:20:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.307+0000 7fce1e59c640 1 --2- 172.21.15.52:0/3851265700 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fce040678c0 0x7fce04069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.311+0000 7fce1e59c640 1 --2- 172.21.15.52:0/3851265700 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fce040678c0 0x7fce04069d80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fce08003cf0 tx=0x7fce080079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:27.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.311+0000 7fce16ffd640 1 -- 172.21.15.52:0/3851265700 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fce18057040 con 0x7fce20144590 2024-05-04T21:20:27.625 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:20:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:27.687+0000 7fce2486a640 1 -- 172.21.15.52:0/3851265700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) -- 0x7fce20062e50 con 0x7fce20144590 2024-05-04T21:20:28.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:28.215+0000 7fce16ffd640 1 -- 172.21.15.52:0/3851265700 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v16338) ==== 144+0+0 (secure 0 0 0) 0x7fce18056940 con 0x7fce20144590 2024-05-04T21:20:28.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:28.219+0000 7fce2486a640 1 -- 172.21.15.52:0/3851265700 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fce040678c0 msgr2=0x7fce04069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:28.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:28.219+0000 7fce2486a640 1 --2- 172.21.15.52:0/3851265700 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fce040678c0 0x7fce04069d80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fce08003cf0 tx=0x7fce080079c0 comp rx=0 tx=0).stop 2024-05-04T21:20:28.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:28.219+0000 7fce2486a640 1 -- 172.21.15.52:0/3851265700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce20144590 msgr2=0x7fce20151f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:28.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:28.219+0000 7fce2486a640 1 --2- 172.21.15.52:0/3851265700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce20144590 0x7fce20151f20 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fce18007e70 tx=0x7fce1800e4b0 comp rx=0 tx=0).stop 2024-05-04T21:20:28.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:28.219+0000 7fce2486a640 1 -- 172.21.15.52:0/3851265700 shutdown_connections 2024-05-04T21:20:28.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:28.219+0000 7fce2486a640 1 --2- 172.21.15.52:0/3851265700 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fce040678c0 0x7fce04069d80 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:28.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:28.219+0000 7fce2486a640 1 --2- 172.21.15.52:0/3851265700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce20144590 0x7fce20151f20 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:28.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:28.219+0000 7fce2486a640 1 --2- 172.21.15.52:0/3851265700 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce201441b0 0x7fce201519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:28.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:28.219+0000 7fce2486a640 1 --2- 172.21.15.52:0/3851265700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce200b6a20 0x7fce201514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:28.227 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:20:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:28.219+0000 7fce2486a640 1 -- 172.21.15.52:0/3851265700 >> 172.21.15.52:0/3851265700 conn(0x7fce200b52d0 msgr2=0x7fce2013a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:28.219+0000 7fce2486a640 1 -- 172.21.15.52:0/3851265700 shutdown_connections 2024-05-04T21:20:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:28.219+0000 7fce2486a640 1 -- 172.21.15.52:0/3851265700 wait complete. 2024-05-04T21:20:28.379 INFO:tasks.cephfs_test_runner:test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) ... FAIL 2024-05-04T21:20:28.380 INFO:tasks.cephfs_test_runner: 2024-05-04T21:20:28.380 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-04T21:20:28.380 INFO:tasks.cephfs_test_runner:FAIL: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-05-04T21:20:28.380 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-04T21:20:28.380 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-04T21:20:28.380 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_09dbd6bba52d19ba471c33bc15c008e6ad158ea6/qa/tasks/cephfs/tests_from_xfstests_dev.py", line 12, in test_generic 2024-05-04T21:20:28.380 INFO:tasks.cephfs_test_runner: self.run_generic_tests() 2024-05-04T21:20:28.380 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_09dbd6bba52d19ba471c33bc15c008e6ad158ea6/qa/tasks/cephfs/xfstests_dev.py", line 407, in run_generic_tests 2024-05-04T21:20:28.380 INFO:tasks.cephfs_test_runner: return self.run_testdir('generic') 2024-05-04T21:20:28.380 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_09dbd6bba52d19ba471c33bc15c008e6ad158ea6/qa/tasks/cephfs/xfstests_dev.py", line 399, in run_testdir 2024-05-04T21:20:28.380 INFO:tasks.cephfs_test_runner: self.assertEqual(self.total_tests_failed, 0) 2024-05-04T21:20:28.380 INFO:tasks.cephfs_test_runner:AssertionError: 16 != 0 2024-05-04T21:20:28.381 INFO:tasks.cephfs_test_runner: 2024-05-04T21:20:28.381 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-04T21:20:28.381 INFO:tasks.cephfs_test_runner:Ran 1 test in 32495.875s 2024-05-04T21:20:28.381 INFO:tasks.cephfs_test_runner: 2024-05-04T21:20:28.381 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2024-05-04T21:20:28.381 INFO:tasks.cephfs_test_runner: 2024-05-04T21:20:28.381 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-04T21:20:28.381 INFO:tasks.cephfs_test_runner:FAIL: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-05-04T21:20:28.381 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-04T21:20:28.381 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-04T21:20:28.381 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_09dbd6bba52d19ba471c33bc15c008e6ad158ea6/qa/tasks/cephfs/tests_from_xfstests_dev.py", line 12, in test_generic 2024-05-04T21:20:28.381 INFO:tasks.cephfs_test_runner: self.run_generic_tests() 2024-05-04T21:20:28.381 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_09dbd6bba52d19ba471c33bc15c008e6ad158ea6/qa/tasks/cephfs/xfstests_dev.py", line 407, in run_generic_tests 2024-05-04T21:20:28.381 INFO:tasks.cephfs_test_runner: return self.run_testdir('generic') 2024-05-04T21:20:28.382 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_09dbd6bba52d19ba471c33bc15c008e6ad158ea6/qa/tasks/cephfs/xfstests_dev.py", line 399, in run_testdir 2024-05-04T21:20:28.382 INFO:tasks.cephfs_test_runner: self.assertEqual(self.total_tests_failed, 0) 2024-05-04T21:20:28.382 INFO:tasks.cephfs_test_runner:AssertionError: 16 != 0 2024-05-04T21:20:28.382 INFO:tasks.cephfs_test_runner: 2024-05-04T21:20:28.382 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_09dbd6bba52d19ba471c33bc15c008e6ad158ea6/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-05-04T21:20:28.622 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=eaaa91d56cfa4917ad2e983a0a07bcfc Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_09dbd6bba52d19ba471c33bc15c008e6ad158ea6/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-05-04T21:20:28.623 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-04T21:20:28.636 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-04T21:20:28.644 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-04T21:20:28.644 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T21:20:28.644 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T21:20:28.658 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16818 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-04T21:20:28.658 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-04T21:20:28.658 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T21:20:28.702 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16818 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-04T21:20:28.703 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T21:20:28.703 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T21:20:28.853 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16191 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-04T21:20:28.854 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-04T21:20:28.854 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T21:20:28.902 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16191 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-04T21:20:28.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T21:20:28.903 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-05-04T21:20:29.112 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-04T21:20:29.113 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-04T21:20:29.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T21:20:29.113 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-04T21:20:29.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T21:20:29.124 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-04T21:20:29.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T21:20:29.185 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2024-05-04T21:20:29.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T21:20:29.364 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-05-04T21:20:29.650 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-04T21:20:29.651 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-04T21:20:29.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T21:20:29.652 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-04T21:20:29.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T21:20:29.670 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T21:20:29.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T21:20:29.747 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-05-04T21:20:29.932 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-04T21:20:29.941 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-04T21:20:29.942 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-04T21:20:30.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.111+0000 7f5105643640 1 Processor -- start 2024-05-04T21:20:30.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.111+0000 7f5105643640 1 -- start start 2024-05-04T21:20:30.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.115+0000 7f5105643640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f51000b3670 0x7f51000cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:30.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.115+0000 7f5105643640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f51000cd200 0x7f5100070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:30.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.115+0000 7f5105643640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f510015e9b0 con 0x7f51001589f0 2024-05-04T21:20:30.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.115+0000 7f5105643640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f5100158dd0 con 0x7f51000cd200 2024-05-04T21:20:30.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.115+0000 7f5105643640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f510015f2e0 con 0x7f51000b3670 2024-05-04T21:20:30.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.115+0000 7f50feffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f51001589f0 0x7f51000b2f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.52:42726/0 (socket says 172.21.15.52:42726) 2024-05-04T21:20:30.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.115+0000 7f50feffd640 1 -- 172.21.15.52:0/3821515897 learned_addr learned my addr 172.21.15.52:0/3821515897 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:30.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.115+0000 7f50ff7fe640 1 --2- 172.21.15.52:0/3821515897 >> v2:172.21.15.52:3300/0 conn(0x7f51000cd200 0x7f5100070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:30.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.115+0000 7f50ff7fe640 1 -- 172.21.15.52:0/3821515897 >> v2:172.21.15.111:3300/0 conn(0x7f51000b3670 msgr2=0x7f51000cccc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T21:20:30.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.115+0000 7f50ff7fe640 1 --2- 172.21.15.52:0/3821515897 >> v2:172.21.15.111:3300/0 conn(0x7f51000b3670 0x7f51000cccc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:30.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.115+0000 7f50ff7fe640 1 -- 172.21.15.52:0/3821515897 >> v1:172.21.15.111:6789/0 conn(0x7f51001589f0 legacy=0x7f51000b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:30.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.115+0000 7f50ff7fe640 1 -- 172.21.15.52:0/3821515897 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510015ec90 con 0x7f51000cd200 2024-05-04T21:20:30.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.119+0000 7f50ff7fe640 1 --2- 172.21.15.52:0/3821515897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51000cd200 0x7f5100070690 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f510015e8a0 tx=0x7f50f401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39833ec3c90f2ef3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:30.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.119+0000 7f50fdffb640 1 -- 172.21.15.52:0/3821515897 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f40b0070 con 0x7f51000cd200 2024-05-04T21:20:30.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.119+0000 7f50fdffb640 1 -- 172.21.15.52:0/3821515897 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f40ab600 con 0x7f51000cd200 2024-05-04T21:20:30.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.119+0000 7f50fdffb640 1 -- 172.21.15.52:0/3821515897 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f40ab9d0 con 0x7f51000cd200 2024-05-04T21:20:30.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.119+0000 7f5105643640 1 -- 172.21.15.52:0/3821515897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51000cd200 msgr2=0x7f5100070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:30.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.119+0000 7f5105643640 1 --2- 172.21.15.52:0/3821515897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51000cd200 0x7f5100070690 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f510015e8a0 tx=0x7f50f401c300 comp rx=0 tx=0).stop 2024-05-04T21:20:30.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.119+0000 7f5105643640 1 -- 172.21.15.52:0/3821515897 shutdown_connections 2024-05-04T21:20:30.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.119+0000 7f5105643640 1 --2- 172.21.15.52:0/3821515897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51000cd200 0x7f5100070690 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:30.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.119+0000 7f5105643640 1 --2- 172.21.15.52:0/3821515897 >> v2:172.21.15.111:3300/0 conn(0x7f51000b3670 0x7f51000cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:30.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.119+0000 7f5105643640 1 -- 172.21.15.52:0/3821515897 >> 172.21.15.52:0/3821515897 conn(0x7f5100067070 msgr2=0x7f5100067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:30.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.119+0000 7f5105643640 1 -- 172.21.15.52:0/3821515897 shutdown_connections 2024-05-04T21:20:30.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.119+0000 7f5105643640 1 -- 172.21.15.52:0/3821515897 wait complete. 2024-05-04T21:20:30.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.123+0000 7f5105643640 1 Processor -- start 2024-05-04T21:20:30.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.123+0000 7f5105643640 1 -- start start 2024-05-04T21:20:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.123+0000 7f5105643640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51000b3670 0x7f5100159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.123+0000 7f5105643640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51000cd200 0x7f51000abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.123+0000 7f5105643640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f51001589f0 0x7f51001f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.123+0000 7f5105643640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f51000cd7d0 con 0x7f51000cd200 2024-05-04T21:20:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.123+0000 7f50fe7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51000cd200 0x7f51000abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.123+0000 7f50feffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51000b3670 0x7f5100159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.123+0000 7f50feffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51000b3670 0x7f5100159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:51626/0 (socket says 172.21.15.52:51626) 2024-05-04T21:20:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.123+0000 7f50fe7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51000cd200 0x7f51000abdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59724/0 (socket says 172.21.15.52:59724) 2024-05-04T21:20:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.123+0000 7f50feffd640 1 -- 172.21.15.52:0/3443930537 learned_addr learned my addr 172.21.15.52:0/3443930537 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.123+0000 7f5105643640 1 -- 172.21.15.52:0/3443930537 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f51000cd650 con 0x7f51000b3670 2024-05-04T21:20:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.123+0000 7f50ff7fe640 1 --2- 172.21.15.52:0/3443930537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f51001589f0 0x7f51001f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.123+0000 7f5105643640 1 -- 172.21.15.52:0/3443930537 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f51000713e0 con 0x7f51001589f0 2024-05-04T21:20:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.127+0000 7f50fe7fc640 1 -- 172.21.15.52:0/3443930537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f51001589f0 msgr2=0x7f51001f27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.127+0000 7f50fe7fc640 1 --2- 172.21.15.52:0/3443930537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f51001589f0 0x7f51001f27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.127+0000 7f50fe7fc640 1 -- 172.21.15.52:0/3443930537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51000b3670 msgr2=0x7f5100159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.127+0000 7f50fe7fc640 1 --2- 172.21.15.52:0/3443930537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51000b3670 0x7f5100159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.127+0000 7f50fe7fc640 1 -- 172.21.15.52:0/3443930537 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51001f3220 con 0x7f51000cd200 2024-05-04T21:20:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.127+0000 7f50fe7fc640 1 --2- 172.21.15.52:0/3443930537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51000cd200 0x7f51000abdc0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f51000d0100 tx=0x7f50f000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.127+0000 7f50e77fe640 1 -- 172.21.15.52:0/3443930537 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0015020 con 0x7f51000cd200 2024-05-04T21:20:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.127+0000 7f50e77fe640 1 -- 172.21.15.52:0/3443930537 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f0011830 con 0x7f51000cd200 2024-05-04T21:20:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.127+0000 7f5105643640 1 -- 172.21.15.52:0/3443930537 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51001f3500 con 0x7f51000cd200 2024-05-04T21:20:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.127+0000 7f5105643640 1 -- 172.21.15.52:0/3443930537 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51000ab8a0 con 0x7f51000cd200 2024-05-04T21:20:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.131+0000 7f50e77fe640 1 -- 172.21.15.52:0/3443930537 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0011c20 con 0x7f51000cd200 2024-05-04T21:20:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.131+0000 7f50e77fe640 1 -- 172.21.15.52:0/3443930537 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f50f0018020 con 0x7f51000cd200 2024-05-04T21:20:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.131+0000 7f50e77fe640 1 --2- 172.21.15.52:0/3443930537 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f50e8067ad0 0x7f50e8069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.131+0000 7f5105643640 1 -- 172.21.15.52:0/3443930537 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51000abaa0 con 0x7f51000cd200 2024-05-04T21:20:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.131+0000 7f50e77fe640 1 -- 172.21.15.52:0/3443930537 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f50f00856f0 con 0x7f51000cd200 2024-05-04T21:20:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.131+0000 7f50feffd640 1 --2- 172.21.15.52:0/3443930537 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f50e8067ad0 0x7f50e8069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.131+0000 7f50feffd640 1 --2- 172.21.15.52:0/3443930537 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f50e8067ad0 0x7f50e8069f90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f50f80012f0 tx=0x7f50f8001d20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:30.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.147+0000 7f50e77fe640 1 -- 172.21.15.52:0/3443930537 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f51000abaa0 con 0x7f51000cd200 2024-05-04T21:20:30.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.431+0000 7f5105643640 1 -- 172.21.15.52:0/3443930537 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5100062e50 con 0x7f50e8067ad0 2024-05-04T21:20:30.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.859+0000 7f50e77fe640 1 -- 172.21.15.52:0/3443930537 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13650 (secure 0 0 0) 0x7f5100062e50 con 0x7f50e8067ad0 2024-05-04T21:20:30.891 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-04T21:20:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.863+0000 7f5105643640 1 -- 172.21.15.52:0/3443930537 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f50e8067ad0 msgr2=0x7f50e8069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:30.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.863+0000 7f5105643640 1 --2- 172.21.15.52:0/3443930537 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f50e8067ad0 0x7f50e8069f90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f50f80012f0 tx=0x7f50f8001d20 comp rx=0 tx=0).stop 2024-05-04T21:20:30.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.863+0000 7f5105643640 1 -- 172.21.15.52:0/3443930537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51000cd200 msgr2=0x7f51000abdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:30.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.863+0000 7f5105643640 1 --2- 172.21.15.52:0/3443930537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51000cd200 0x7f51000abdc0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f51000d0100 tx=0x7f50f000b870 comp rx=0 tx=0).stop 2024-05-04T21:20:30.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.863+0000 7f5105643640 1 -- 172.21.15.52:0/3443930537 shutdown_connections 2024-05-04T21:20:30.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.863+0000 7f5105643640 1 --2- 172.21.15.52:0/3443930537 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f50e8067ad0 0x7f50e8069f90 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:30.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.863+0000 7f5105643640 1 --2- 172.21.15.52:0/3443930537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f51001589f0 0x7f51001f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:30.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.863+0000 7f5105643640 1 --2- 172.21.15.52:0/3443930537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51000cd200 0x7f51000abdc0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:30.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.863+0000 7f5105643640 1 --2- 172.21.15.52:0/3443930537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51000b3670 0x7f5100159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:30.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.863+0000 7f5105643640 1 -- 172.21.15.52:0/3443930537 >> 172.21.15.52:0/3443930537 conn(0x7f5100067070 msgr2=0x7f51000717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:30.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.863+0000 7f5105643640 1 -- 172.21.15.52:0/3443930537 shutdown_connections 2024-05-04T21:20:30.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:30.863+0000 7f5105643640 1 -- 172.21.15.52:0/3443930537 wait complete. 2024-05-04T21:20:30.894 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":16294,"stamp":"2024-05-04T21:20:28.455509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":56081928,"kb_used_data":43994408,"kb_used_omap":4601,"kb_used_meta":12082822,"kb_avail":698892792,"statfs":{"total":773094113280,"available":715666219008,"internally_reserved":0,"allocated":45050273792,"data_stored":45044454344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4712188,"internal_metadata":12372809988},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23289006641,"num_objects":-12776,"num_object_clones":0,"num_object_copies":-25552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-12776,"num_whiteouts":0,"num_read":-1898495,"num_read_kb":-132833473,"num_write":-7540314,"num_write_kb":-203294643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-183,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001307"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":465,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-04T21:20:23.445606+0000","last_change":"2024-05-04T12:18:12.644577+0000","last_active":"2024-05-04T21:20:23.445606+0000","last_peered":"2024-05-04T21:20:23.445606+0000","last_clean":"2024-05-04T21:20:23.445606+0000","last_became_active":"2024-05-04T12:18:12.640041+0000","last_became_peered":"2024-05-04T12:18:12.640041+0000","last_unstale":"2024-05-04T21:20:23.445606+0000","last_undegraded":"2024-05-04T21:20:23.445606+0000","last_fullsized":"2024-05-04T21:20:23.445606+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_clean_scrub_stamp":"2024-05-04T12:18:11.260157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:15:02.089387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834581359,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7208892,"kb_used_data":5603248,"kb_used_omap":21,"kb_used_meta":1605610,"kb_avail":87162948,"statfs":{"total":96636764160,"available":89254858752,"internally_reserved":0,"allocated":5737725952,"data_stored":5736995591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22500,"internal_metadata":1644144668},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834581359,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5626836,"kb_used_data":3989832,"kb_used_omap":796,"kb_used_meta":1636195,"kb_avail":88745004,"statfs":{"total":96636764160,"available":90874884096,"internally_reserved":0,"allocated":4085587968,"data_stored":4084921332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":815583,"internal_metadata":1675464225},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834581358,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7872044,"kb_used_data":6229984,"kb_used_omap":542,"kb_used_meta":1641505,"kb_avail":86499796,"statfs":{"total":96636764160,"available":88575791104,"internally_reserved":0,"allocated":6379503616,"data_stored":6378777374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":555581,"internal_metadata":1680901571},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834581358,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8145188,"kb_used_data":6538200,"kb_used_omap":836,"kb_used_meta":1606139,"kb_avail":86226652,"statfs":{"total":96636764160,"available":88296091648,"internally_reserved":0,"allocated":6695116800,"data_stored":6694227703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":856296,"internal_metadata":1644687128},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834581346,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8169176,"kb_used_data":6579404,"kb_used_omap":888,"kb_used_meta":1588871,"kb_avail":86202664,"statfs":{"total":96636764160,"available":88271527936,"internally_reserved":0,"allocated":6737309696,"data_stored":6736502922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":909648,"internal_metadata":1627004592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834581351,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5655756,"kb_used_data":4406528,"kb_used_omap":363,"kb_used_meta":1248852,"kb_avail":88716084,"statfs":{"total":96636764160,"available":90845270016,"internally_reserved":0,"allocated":4512284672,"data_stored":4511656588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":372172,"internal_metadata":1278825012},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834581348,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7883216,"kb_used_data":6561412,"kb_used_omap":926,"kb_used_meta":1320865,"kb_avail":86488624,"statfs":{"total":96636764160,"available":88564350976,"internally_reserved":0,"allocated":6718885888,"data_stored":6718124859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":948664,"internal_metadata":1352566344},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834581354,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5520820,"kb_used_data":4085800,"kb_used_omap":226,"kb_used_meta":1434781,"kb_avail":88851020,"statfs":{"total":96636764160,"available":90983444480,"internally_reserved":0,"allocated":4183859200,"data_stored":4183247975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":231744,"internal_metadata":1469216448},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T21:20:30.895 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-04T21:20:31.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.031+0000 7f624584c640 1 Processor -- start 2024-05-04T21:20:31.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.031+0000 7f624584c640 1 -- start start 2024-05-04T21:20:31.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.031+0000 7f624584c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f624015d1d0 0x7f6240070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.031+0000 7f624584c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6240070bd0 0x7f6240072020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.031+0000 7f624584c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6240071070 con 0x7f6240072560 2024-05-04T21:20:31.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.031+0000 7f624584c640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f62400729b0 con 0x7f624015d1d0 2024-05-04T21:20:31.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.031+0000 7f624584c640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f6240071350 con 0x7f6240070bd0 2024-05-04T21:20:31.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.031+0000 7f623effd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f624015d1d0 0x7f6240070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.031+0000 7f623effd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f624015d1d0 0x7f6240070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59732/0 (socket says 172.21.15.52:59732) 2024-05-04T21:20:31.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.031+0000 7f623effd640 1 -- 172.21.15.52:0/249100057 learned_addr learned my addr 172.21.15.52:0/249100057 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:31.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f623e7fc640 1 --2- 172.21.15.52:0/249100057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6240070bd0 0x7f6240072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f623effd640 1 -- 172.21.15.52:0/249100057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6240070bd0 msgr2=0x7f6240072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:31.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f623effd640 1 --2- 172.21.15.52:0/249100057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6240070bd0 0x7f6240072020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f623effd640 1 -- 172.21.15.52:0/249100057 >> v1:172.21.15.111:6789/0 conn(0x7f6240072560 legacy=0x7f624015fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f623effd640 1 -- 172.21.15.52:0/249100057 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6240071630 con 0x7f624015d1d0 2024-05-04T21:20:31.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f623effd640 1 --2- 172.21.15.52:0/249100057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f624015d1d0 0x7f6240070690 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f6230009870 tx=0x7f623001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b7c77ce634fb1d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:31.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f623dffb640 1 -- 172.21.15.52:0/249100057 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62300b0070 con 0x7f624015d1d0 2024-05-04T21:20:31.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f623dffb640 1 -- 172.21.15.52:0/249100057 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62300ab600 con 0x7f624015d1d0 2024-05-04T21:20:31.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f623dffb640 1 -- 172.21.15.52:0/249100057 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62300ab9d0 con 0x7f624015d1d0 2024-05-04T21:20:31.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f624584c640 1 -- 172.21.15.52:0/249100057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f624015d1d0 msgr2=0x7f6240070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f624584c640 1 --2- 172.21.15.52:0/249100057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f624015d1d0 0x7f6240070690 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f6230009870 tx=0x7f623001c4d0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f624584c640 1 -- 172.21.15.52:0/249100057 shutdown_connections 2024-05-04T21:20:31.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f624584c640 1 --2- 172.21.15.52:0/249100057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6240070bd0 0x7f6240072020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f624584c640 1 --2- 172.21.15.52:0/249100057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f624015d1d0 0x7f6240070690 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f624584c640 1 -- 172.21.15.52:0/249100057 >> 172.21.15.52:0/249100057 conn(0x7f62400b2fe0 msgr2=0x7f62400b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:31.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f624584c640 1 -- 172.21.15.52:0/249100057 shutdown_connections 2024-05-04T21:20:31.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.035+0000 7f624584c640 1 -- 172.21.15.52:0/249100057 wait complete. 2024-05-04T21:20:31.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f624584c640 1 Processor -- start 2024-05-04T21:20:31.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f624584c640 1 -- start start 2024-05-04T21:20:31.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f624584c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6240070bd0 0x7f6240136000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f624584c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6240072560 0x7f6240137f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f624584c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f624015d1d0 0x7f62400aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f624584c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6240169240 con 0x7f6240070bd0 2024-05-04T21:20:31.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f624584c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f62401690c0 con 0x7f624015d1d0 2024-05-04T21:20:31.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f624584c640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f62401693c0 con 0x7f6240072560 2024-05-04T21:20:31.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f623effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6240070bd0 0x7f6240136000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f623effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6240070bd0 0x7f6240136000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59734/0 (socket says 172.21.15.52:59734) 2024-05-04T21:20:31.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f623effd640 1 -- 172.21.15.52:0/885190078 learned_addr learned my addr 172.21.15.52:0/885190078 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:31.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f623e7fc640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6240072560 0x7f6240137f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f623effd640 1 -- 172.21.15.52:0/885190078 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6240072560 msgr2=0x7f6240137f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f623effd640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6240072560 0x7f6240137f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f623effd640 1 -- 172.21.15.52:0/885190078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f624015d1d0 msgr2=0x7f62400aa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f623f7fe640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f624015d1d0 0x7f62400aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f623effd640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f624015d1d0 0x7f62400aa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f623effd640 1 -- 172.21.15.52:0/885190078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62401386e0 con 0x7f6240070bd0 2024-05-04T21:20:31.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f623e7fc640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6240072560 0x7f6240137f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T21:20:31.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f623f7fe640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f624015d1d0 0x7f62400aa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:31.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f623effd640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6240070bd0 0x7f6240136000 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f623001ab80 tx=0x7f6230019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:31.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f6227fff640 1 -- 172.21.15.52:0/885190078 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62300ba070 con 0x7f6240070bd0 2024-05-04T21:20:31.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f6227fff640 1 -- 172.21.15.52:0/885190078 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62300b4420 con 0x7f6240070bd0 2024-05-04T21:20:31.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f624584c640 1 -- 172.21.15.52:0/885190078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62401389c0 con 0x7f6240070bd0 2024-05-04T21:20:31.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f6227fff640 1 -- 172.21.15.52:0/885190078 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62300b0040 con 0x7f6240070bd0 2024-05-04T21:20:31.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.039+0000 7f624584c640 1 -- 172.21.15.52:0/885190078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6240138f80 con 0x7f6240070bd0 2024-05-04T21:20:31.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.043+0000 7f624584c640 1 -- 172.21.15.52:0/885190078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6240062e50 con 0x7f6240070bd0 2024-05-04T21:20:31.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.043+0000 7f6227fff640 1 -- 172.21.15.52:0/885190078 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f62300bd070 con 0x7f6240070bd0 2024-05-04T21:20:31.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.043+0000 7f6227fff640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f622c0678c0 0x7f622c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.043+0000 7f623e7fc640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f622c0678c0 0x7f622c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.043+0000 7f6227fff640 1 -- 172.21.15.52:0/885190078 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f623011e9f0 con 0x7f6240070bd0 2024-05-04T21:20:31.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.043+0000 7f623e7fc640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f622c0678c0 0x7f622c069d80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6240153240 tx=0x7f6234008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:31.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.047+0000 7f6227fff640 1 -- 172.21.15.52:0/885190078 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6230016150 con 0x7f6240070bd0 2024-05-04T21:20:31.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.251+0000 7f624584c640 1 -- 172.21.15.52:0/885190078 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f62400af8d0 con 0x7f622c0678c0 2024-05-04T21:20:31.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.251+0000 7f6227fff640 1 -- 172.21.15.52:0/885190078 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13666 (secure 0 0 0) 0x7f62400af8d0 con 0x7f622c0678c0 2024-05-04T21:20:31.255 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:31.256 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-04T21:20:31.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.255+0000 7f624584c640 1 -- 172.21.15.52:0/885190078 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f622c0678c0 msgr2=0x7f622c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.255+0000 7f624584c640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f622c0678c0 0x7f622c069d80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6240153240 tx=0x7f6234008040 comp rx=0 tx=0).stop 2024-05-04T21:20:31.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.255+0000 7f624584c640 1 -- 172.21.15.52:0/885190078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6240070bd0 msgr2=0x7f6240136000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.255+0000 7f624584c640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6240070bd0 0x7f6240136000 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f623001ab80 tx=0x7f6230019270 comp rx=0 tx=0).stop 2024-05-04T21:20:31.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.255+0000 7f624584c640 1 -- 172.21.15.52:0/885190078 shutdown_connections 2024-05-04T21:20:31.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.255+0000 7f624584c640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f622c0678c0 0x7f622c069d80 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.255+0000 7f624584c640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f624015d1d0 0x7f62400aa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.255+0000 7f624584c640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6240072560 0x7f6240137f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.255+0000 7f624584c640 1 --2- 172.21.15.52:0/885190078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6240070bd0 0x7f6240136000 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.255+0000 7f624584c640 1 -- 172.21.15.52:0/885190078 >> 172.21.15.52:0/885190078 conn(0x7f62400b2fe0 msgr2=0x7f624015eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:31.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.255+0000 7f624584c640 1 -- 172.21.15.52:0/885190078 shutdown_connections 2024-05-04T21:20:31.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.255+0000 7f624584c640 1 -- 172.21.15.52:0/885190078 wait complete. 2024-05-04T21:20:31.281 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":16295,"stamp":"2024-05-04T21:20:30.861611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":29400156,"kb_used_data":17550604,"kb_used_omap":4642,"kb_used_meta":11844829,"kb_avail":725574564,"statfs":{"total":773094113280,"available":742988353536,"internally_reserved":0,"allocated":17971818496,"data_stored":17968310595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4754182,"internal_metadata":12129105146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":478,"apply_latency_ms":478,"commit_latency_ns":478000000,"apply_latency_ns":478000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23289006641,"num_objects":-12776,"num_object_clones":0,"num_object_copies":-25552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-12776,"num_whiteouts":0,"num_read":-1898495,"num_read_kb":-132833473,"num_write":-7540314,"num_write_kb":-203294643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-183,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.542911"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":469,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-04T21:20:26.282615+0000","last_change":"2024-05-04T12:18:12.644577+0000","last_active":"2024-05-04T21:20:26.282615+0000","last_peered":"2024-05-04T21:20:26.282615+0000","last_clean":"2024-05-04T21:20:26.282615+0000","last_became_active":"2024-05-04T12:18:12.640041+0000","last_became_peered":"2024-05-04T12:18:12.640041+0000","last_unstale":"2024-05-04T21:20:26.282615+0000","last_undegraded":"2024-05-04T21:20:26.282615+0000","last_fullsized":"2024-05-04T21:20:26.282615+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_clean_scrub_stamp":"2024-05-04T12:18:11.260157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:15:02.089387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834581360,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1654068,"kb_used_data":1000,"kb_used_omap":21,"kb_used_meta":1653034,"kb_avail":92717772,"statfs":{"total":96636764160,"available":94942998528,"internally_reserved":0,"allocated":1024000,"data_stored":759466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22500,"internal_metadata":1692706844},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":86,"apply_latency_ms":86,"commit_latency_ns":86000000,"apply_latency_ns":86000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834581360,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1637420,"kb_used_data":420,"kb_used_omap":796,"kb_used_meta":1636195,"kb_avail":92734420,"statfs":{"total":96636764160,"available":94960046080,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":815583,"internal_metadata":1675464225},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":65,"apply_latency_ms":65,"commit_latency_ns":65000000,"apply_latency_ns":65000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834581359,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1498156,"kb_used_data":420,"kb_used_omap":583,"kb_used_meta":1497144,"kb_avail":92873684,"statfs":{"total":96636764160,"available":95102652416,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":597575,"internal_metadata":1533075897},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":91,"apply_latency_ms":91,"commit_latency_ns":91000000,"apply_latency_ns":91000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834581359,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1462188,"kb_used_data":420,"kb_used_omap":836,"kb_used_meta":1460923,"kb_avail":92909652,"statfs":{"total":96636764160,"available":95139483648,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":856296,"internal_metadata":1495985944},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":109,"apply_latency_ms":109,"commit_latency_ns":109000000,"apply_latency_ns":109000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834581346,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8169176,"kb_used_data":6579404,"kb_used_omap":888,"kb_used_meta":1588871,"kb_avail":86202664,"statfs":{"total":96636764160,"available":88271527936,"internally_reserved":0,"allocated":6737309696,"data_stored":6736502922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":909648,"internal_metadata":1627004592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834581351,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5655756,"kb_used_data":4406528,"kb_used_omap":363,"kb_used_meta":1248852,"kb_avail":88716084,"statfs":{"total":96636764160,"available":90845270016,"internally_reserved":0,"allocated":4512284672,"data_stored":4511656588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":372172,"internal_metadata":1278825012},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834581348,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7883216,"kb_used_data":6561412,"kb_used_omap":926,"kb_used_meta":1320865,"kb_avail":86488624,"statfs":{"total":96636764160,"available":88564350976,"internally_reserved":0,"allocated":6718885888,"data_stored":6718124859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":948664,"internal_metadata":1352566344},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834581355,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1440176,"kb_used_data":1000,"kb_used_omap":226,"kb_used_meta":1438941,"kb_avail":92931664,"statfs":{"total":96636764160,"available":95162023936,"internally_reserved":0,"allocated":1024000,"data_stored":759466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":231744,"internal_metadata":1473476288},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":127,"apply_latency_ms":127,"commit_latency_ns":127000000,"apply_latency_ns":127000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T21:20:31.281 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-04T21:20:31.282 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-04T21:20:31.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.435+0000 7fd8b668f640 1 Processor -- start 2024-05-04T21:20:31.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.435+0000 7fd8b668f640 1 -- start start 2024-05-04T21:20:31.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8b668f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd8b00d0480 0x7fd8b0142170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8b668f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8b0160220 0x7fd8b00d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8b668f640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b013f6d0 con 0x7fd8b00a9480 2024-05-04T21:20:31.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8b668f640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b01426b0 con 0x7fd8b00d0480 2024-05-04T21:20:31.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8b668f640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b00a9910 con 0x7fd8b0160220 2024-05-04T21:20:31.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8a7fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fd8b00a9480 0x7fd8b015fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55444/0 (socket says 172.21.15.52:55444) 2024-05-04T21:20:31.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8a7fff640 1 -- 172.21.15.52:0/471587425 learned_addr learned my addr 172.21.15.52:0/471587425 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:31.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8affff640 1 --2- 172.21.15.52:0/471587425 >> v2:172.21.15.52:3300/0 conn(0x7fd8b00d0480 0x7fd8b0142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8b4c2c640 1 --2- 172.21.15.52:0/471587425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8b0160220 0x7fd8b00d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8af7fe640 1 -- 172.21.15.52:0/471587425 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2384903525 0 0) 0x7fd8b013f6d0 con 0x7fd8b00a9480 2024-05-04T21:20:31.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8af7fe640 1 -- 172.21.15.52:0/471587425 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd89c003530 con 0x7fd8b00a9480 2024-05-04T21:20:31.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8affff640 1 -- 172.21.15.52:0/471587425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8b0160220 msgr2=0x7fd8b00d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:31.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8affff640 1 --2- 172.21.15.52:0/471587425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8b0160220 0x7fd8b00d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8affff640 1 -- 172.21.15.52:0/471587425 >> v1:172.21.15.52:6789/0 conn(0x7fd8b00a9480 legacy=0x7fd8b015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8af7fe640 1 -- 172.21.15.52:0/471587425 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2444754313 0 0) 0x7fd89c003530 con 0x7fd8b00a9480 2024-05-04T21:20:31.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.439+0000 7fd8affff640 1 -- 172.21.15.52:0/471587425 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b00a9bf0 con 0x7fd8b00d0480 2024-05-04T21:20:31.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.447+0000 7fd8affff640 1 --2- 172.21.15.52:0/471587425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8b00d0480 0x7fd8b0142170 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00097c0 tx=0x7fd8a00207d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82aece233a32b13 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:31.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.447+0000 7fd8af7fe640 1 -- 172.21.15.52:0/471587425 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898001680 con 0x7fd8b00d0480 2024-05-04T21:20:31.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.447+0000 7fd8af7fe640 1 -- 172.21.15.52:0/471587425 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a00ac430 con 0x7fd8b00d0480 2024-05-04T21:20:31.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.447+0000 7fd8af7fe640 1 -- 172.21.15.52:0/471587425 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00ac600 con 0x7fd8b00d0480 2024-05-04T21:20:31.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.447+0000 7fd8b668f640 1 -- 172.21.15.52:0/471587425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8b00d0480 msgr2=0x7fd8b0142170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.447+0000 7fd8b668f640 1 --2- 172.21.15.52:0/471587425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8b00d0480 0x7fd8b0142170 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00097c0 tx=0x7fd8a00207d0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.447+0000 7fd8b668f640 1 -- 172.21.15.52:0/471587425 shutdown_connections 2024-05-04T21:20:31.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.447+0000 7fd8b668f640 1 --2- 172.21.15.52:0/471587425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8b0160220 0x7fd8b00d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.447+0000 7fd8b668f640 1 --2- 172.21.15.52:0/471587425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8b00d0480 0x7fd8b0142170 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.447+0000 7fd8b668f640 1 -- 172.21.15.52:0/471587425 >> 172.21.15.52:0/471587425 conn(0x7fd8b0070690 msgr2=0x7fd8b0070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:31.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.447+0000 7fd8b668f640 1 -- 172.21.15.52:0/471587425 shutdown_connections 2024-05-04T21:20:31.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.447+0000 7fd8b668f640 1 -- 172.21.15.52:0/471587425 wait complete. 2024-05-04T21:20:31.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b668f640 1 Processor -- start 2024-05-04T21:20:31.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b668f640 1 -- start start 2024-05-04T21:20:31.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b668f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8b00d0480 0x7fd8b00b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b668f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8b0160220 0x7fd8b0154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b668f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8b01554c0 0x7fd8b01f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b4c2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8b01554c0 0x7fd8b01f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b4c2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8b01554c0 0x7fd8b01f6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59754/0 (socket says 172.21.15.52:59754) 2024-05-04T21:20:31.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b4c2c640 1 -- 172.21.15.52:0/177814463 learned_addr learned my addr 172.21.15.52:0/177814463 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:31.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b668f640 1 -- 172.21.15.52:0/177814463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b00d1d60 con 0x7fd8b01554c0 2024-05-04T21:20:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8affff640 1 --2- 172.21.15.52:0/177814463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8b00d0480 0x7fd8b00b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b668f640 1 -- 172.21.15.52:0/177814463 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b0160900 con 0x7fd8b00d0480 2024-05-04T21:20:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b668f640 1 -- 172.21.15.52:0/177814463 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b00d1ee0 con 0x7fd8b0160220 2024-05-04T21:20:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b4c2c640 1 -- 172.21.15.52:0/177814463 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8b0160220 msgr2=0x7fd8b0154f80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-04T21:20:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b4c2c640 1 --2- 172.21.15.52:0/177814463 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8b0160220 0x7fd8b0154f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b4c2c640 1 -- 172.21.15.52:0/177814463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8b00d0480 msgr2=0x7fd8b00b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b4c2c640 1 --2- 172.21.15.52:0/177814463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8b00d0480 0x7fd8b00b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b4c2c640 1 -- 172.21.15.52:0/177814463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b0155940 con 0x7fd8b01554c0 2024-05-04T21:20:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8b4c2c640 1 --2- 172.21.15.52:0/177814463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8b01554c0 0x7fd8b01f6b40 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800efc0 tx=0x7fd8a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.451+0000 7fd8ad7fa640 1 -- 172.21.15.52:0/177814463 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8018020 con 0x7fd8b01554c0 2024-05-04T21:20:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.455+0000 7fd8ad7fa640 1 -- 172.21.15.52:0/177814463 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a8014480 con 0x7fd8b01554c0 2024-05-04T21:20:31.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.455+0000 7fd8ad7fa640 1 -- 172.21.15.52:0/177814463 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8014870 con 0x7fd8b01554c0 2024-05-04T21:20:31.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.455+0000 7fd8b668f640 1 -- 172.21.15.52:0/177814463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b013d150 con 0x7fd8b01554c0 2024-05-04T21:20:31.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.455+0000 7fd8b668f640 1 -- 172.21.15.52:0/177814463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b00affa0 con 0x7fd8b01554c0 2024-05-04T21:20:31.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.455+0000 7fd8b668f640 1 -- 172.21.15.52:0/177814463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b0062e50 con 0x7fd8b01554c0 2024-05-04T21:20:31.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.459+0000 7fd8ad7fa640 1 -- 172.21.15.52:0/177814463 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd8a801b020 con 0x7fd8b01554c0 2024-05-04T21:20:31.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.459+0000 7fd8ad7fa640 1 --2- 172.21.15.52:0/177814463 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd890067910 0x7fd890069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.459+0000 7fd8ad7fa640 1 -- 172.21.15.52:0/177814463 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fd8a8086270 con 0x7fd8b01554c0 2024-05-04T21:20:31.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.459+0000 7fd8affff640 1 --2- 172.21.15.52:0/177814463 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd890067910 0x7fd890069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.463+0000 7fd8affff640 1 --2- 172.21.15.52:0/177814463 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd890067910 0x7fd890069dd0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00b3000 tx=0x7fd8a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:31.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.467+0000 7fd8ad7fa640 1 -- 172.21.15.52:0/177814463 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd8a8022070 con 0x7fd8b01554c0 2024-05-04T21:20:31.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.687+0000 7fd8b668f640 1 -- 172.21.15.52:0/177814463 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd8b00715c0 con 0x7fd890067910 2024-05-04T21:20:31.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.687+0000 7fd8ad7fa640 1 -- 172.21.15.52:0/177814463 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13666 (secure 0 0 0) 0x7fd8b00715c0 con 0x7fd890067910 2024-05-04T21:20:31.693 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:31.693 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-04T21:20:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.691+0000 7fd8b668f640 1 -- 172.21.15.52:0/177814463 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd890067910 msgr2=0x7fd890069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.691+0000 7fd8b668f640 1 --2- 172.21.15.52:0/177814463 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd890067910 0x7fd890069dd0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00b3000 tx=0x7fd8a0008040 comp rx=0 tx=0).stop 2024-05-04T21:20:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.691+0000 7fd8b668f640 1 -- 172.21.15.52:0/177814463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8b01554c0 msgr2=0x7fd8b01f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.691+0000 7fd8b668f640 1 --2- 172.21.15.52:0/177814463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8b01554c0 0x7fd8b01f6b40 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800efc0 tx=0x7fd8a800c490 comp rx=0 tx=0).stop 2024-05-04T21:20:31.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.691+0000 7fd8b668f640 1 -- 172.21.15.52:0/177814463 shutdown_connections 2024-05-04T21:20:31.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.691+0000 7fd8b668f640 1 --2- 172.21.15.52:0/177814463 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd890067910 0x7fd890069dd0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.691+0000 7fd8b668f640 1 --2- 172.21.15.52:0/177814463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8b01554c0 0x7fd8b01f6b40 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.691+0000 7fd8b668f640 1 --2- 172.21.15.52:0/177814463 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8b0160220 0x7fd8b0154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.691+0000 7fd8b668f640 1 --2- 172.21.15.52:0/177814463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8b00d0480 0x7fd8b00b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.691+0000 7fd8b668f640 1 -- 172.21.15.52:0/177814463 >> 172.21.15.52:0/177814463 conn(0x7fd8b0070690 msgr2=0x7fd8b0140c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:31.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.691+0000 7fd8b668f640 1 -- 172.21.15.52:0/177814463 shutdown_connections 2024-05-04T21:20:31.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.691+0000 7fd8b668f640 1 -- 172.21.15.52:0/177814463 wait complete. 2024-05-04T21:20:31.722 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":16295,"stamp":"2024-05-04T21:20:30.861611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":29400156,"kb_used_data":17550604,"kb_used_omap":4642,"kb_used_meta":11844829,"kb_avail":725574564,"statfs":{"total":773094113280,"available":742988353536,"internally_reserved":0,"allocated":17971818496,"data_stored":17968310595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4754182,"internal_metadata":12129105146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":478,"apply_latency_ms":478,"commit_latency_ns":478000000,"apply_latency_ns":478000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23289006641,"num_objects":-12776,"num_object_clones":0,"num_object_copies":-25552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-12776,"num_whiteouts":0,"num_read":-1898495,"num_read_kb":-132833473,"num_write":-7540314,"num_write_kb":-203294643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-183,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.542911"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":469,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-04T21:20:26.282615+0000","last_change":"2024-05-04T12:18:12.644577+0000","last_active":"2024-05-04T21:20:26.282615+0000","last_peered":"2024-05-04T21:20:26.282615+0000","last_clean":"2024-05-04T21:20:26.282615+0000","last_became_active":"2024-05-04T12:18:12.640041+0000","last_became_peered":"2024-05-04T12:18:12.640041+0000","last_unstale":"2024-05-04T21:20:26.282615+0000","last_undegraded":"2024-05-04T21:20:26.282615+0000","last_fullsized":"2024-05-04T21:20:26.282615+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_clean_scrub_stamp":"2024-05-04T12:18:11.260157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:15:02.089387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834581360,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1654068,"kb_used_data":1000,"kb_used_omap":21,"kb_used_meta":1653034,"kb_avail":92717772,"statfs":{"total":96636764160,"available":94942998528,"internally_reserved":0,"allocated":1024000,"data_stored":759466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22500,"internal_metadata":1692706844},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":86,"apply_latency_ms":86,"commit_latency_ns":86000000,"apply_latency_ns":86000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834581360,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1637420,"kb_used_data":420,"kb_used_omap":796,"kb_used_meta":1636195,"kb_avail":92734420,"statfs":{"total":96636764160,"available":94960046080,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":815583,"internal_metadata":1675464225},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":65,"apply_latency_ms":65,"commit_latency_ns":65000000,"apply_latency_ns":65000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834581359,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1498156,"kb_used_data":420,"kb_used_omap":583,"kb_used_meta":1497144,"kb_avail":92873684,"statfs":{"total":96636764160,"available":95102652416,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":597575,"internal_metadata":1533075897},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":91,"apply_latency_ms":91,"commit_latency_ns":91000000,"apply_latency_ns":91000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834581359,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1462188,"kb_used_data":420,"kb_used_omap":836,"kb_used_meta":1460923,"kb_avail":92909652,"statfs":{"total":96636764160,"available":95139483648,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":856296,"internal_metadata":1495985944},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":109,"apply_latency_ms":109,"commit_latency_ns":109000000,"apply_latency_ns":109000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834581346,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8169176,"kb_used_data":6579404,"kb_used_omap":888,"kb_used_meta":1588871,"kb_avail":86202664,"statfs":{"total":96636764160,"available":88271527936,"internally_reserved":0,"allocated":6737309696,"data_stored":6736502922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":909648,"internal_metadata":1627004592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834581351,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5655756,"kb_used_data":4406528,"kb_used_omap":363,"kb_used_meta":1248852,"kb_avail":88716084,"statfs":{"total":96636764160,"available":90845270016,"internally_reserved":0,"allocated":4512284672,"data_stored":4511656588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":372172,"internal_metadata":1278825012},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834581348,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7883216,"kb_used_data":6561412,"kb_used_omap":926,"kb_used_meta":1320865,"kb_avail":86488624,"statfs":{"total":96636764160,"available":88564350976,"internally_reserved":0,"allocated":6718885888,"data_stored":6718124859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":948664,"internal_metadata":1352566344},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834581355,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1440176,"kb_used_data":1000,"kb_used_omap":226,"kb_used_meta":1438941,"kb_avail":92931664,"statfs":{"total":96636764160,"available":95162023936,"internally_reserved":0,"allocated":1024000,"data_stored":759466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":231744,"internal_metadata":1473476288},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":127,"apply_latency_ms":127,"commit_latency_ns":127000000,"apply_latency_ns":127000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T21:20:31.723 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T21:20:31.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.835+0000 7fc87eecc640 1 Processor -- start 2024-05-04T21:20:31.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.835+0000 7fc87eecc640 1 -- start start 2024-05-04T21:20:31.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.835+0000 7fc87eecc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc878147f60 0x7fc8780c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.835+0000 7fc87eecc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc8780cfa50 0x7fc878162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.835+0000 7fc87eecc640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc878162b60 con 0x7fc8780d2320 2024-05-04T21:20:31.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.835+0000 7fc87eecc640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc878162e40 con 0x7fc878147f60 2024-05-04T21:20:31.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.835+0000 7fc87eecc640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fc878163120 con 0x7fc8780cfa50 2024-05-04T21:20:31.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.835+0000 7fc87cc68640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc878147f60 0x7fc8780c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.835+0000 7fc87d469640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc8780d2320 0x7fc8780cd7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55454/0 (socket says 172.21.15.52:55454) 2024-05-04T21:20:31.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.835+0000 7fc87cc68640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc878147f60 0x7fc8780c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59762/0 (socket says 172.21.15.52:59762) 2024-05-04T21:20:31.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.835+0000 7fc87d469640 1 -- 172.21.15.52:0/1921787160 learned_addr learned my addr 172.21.15.52:0/1921787160 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:31.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.839+0000 7fc877fff640 1 --2- 172.21.15.52:0/1921787160 >> v2:172.21.15.111:3300/0 conn(0x7fc8780cfa50 0x7fc878162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.839+0000 7fc87cc68640 1 -- 172.21.15.52:0/1921787160 >> v2:172.21.15.111:3300/0 conn(0x7fc8780cfa50 msgr2=0x7fc878162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:31.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.839+0000 7fc87cc68640 1 --2- 172.21.15.52:0/1921787160 >> v2:172.21.15.111:3300/0 conn(0x7fc8780cfa50 0x7fc878162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.839+0000 7fc87cc68640 1 -- 172.21.15.52:0/1921787160 >> v1:172.21.15.52:6789/0 conn(0x7fc8780d2320 legacy=0x7fc8780cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.839+0000 7fc8777fe640 1 -- 172.21.15.52:0/1921787160 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1364988267 0 0) 0x7fc878162b60 con 0x7fc8780d2320 2024-05-04T21:20:31.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.839+0000 7fc87cc68640 1 -- 172.21.15.52:0/1921787160 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8780cd280 con 0x7fc878147f60 2024-05-04T21:20:31.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.839+0000 7fc87cc68640 1 --2- 172.21.15.52:0/1921787160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc878147f60 0x7fc8780c7f80 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fc87806cc70 tx=0x7fc86c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d8ec01a97950e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:31.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.839+0000 7fc8777fe640 1 -- 172.21.15.52:0/1921787160 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0b0070 con 0x7fc878147f60 2024-05-04T21:20:31.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.839+0000 7fc8777fe640 1 -- 172.21.15.52:0/1921787160 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc86c0ab600 con 0x7fc878147f60 2024-05-04T21:20:31.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.839+0000 7fc8777fe640 1 -- 172.21.15.52:0/1921787160 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0ab9d0 con 0x7fc878147f60 2024-05-04T21:20:31.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 -- 172.21.15.52:0/1921787160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc878147f60 msgr2=0x7fc8780c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 --2- 172.21.15.52:0/1921787160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc878147f60 0x7fc8780c7f80 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fc87806cc70 tx=0x7fc86c01c3f0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 -- 172.21.15.52:0/1921787160 shutdown_connections 2024-05-04T21:20:31.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 --2- 172.21.15.52:0/1921787160 >> v2:172.21.15.111:3300/0 conn(0x7fc8780cfa50 0x7fc878162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 --2- 172.21.15.52:0/1921787160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc878147f60 0x7fc8780c7f80 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 -- 172.21.15.52:0/1921787160 >> 172.21.15.52:0/1921787160 conn(0x7fc87806b210 msgr2=0x7fc87806b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:31.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 -- 172.21.15.52:0/1921787160 shutdown_connections 2024-05-04T21:20:31.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 -- 172.21.15.52:0/1921787160 wait complete. 2024-05-04T21:20:31.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 Processor -- start 2024-05-04T21:20:31.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 -- start start 2024-05-04T21:20:31.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8780cfa50 0x7fc87813c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8780d2320 0x7fc878146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87813d000 0x7fc878147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781484c0 con 0x7fc87813d000 2024-05-04T21:20:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc878148340 con 0x7fc8780d2320 2024-05-04T21:20:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc8780ce2c0 con 0x7fc8780cfa50 2024-05-04T21:20:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87d469640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87813d000 0x7fc878147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87d469640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87813d000 0x7fc878147430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59774/0 (socket says 172.21.15.52:59774) 2024-05-04T21:20:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87d469640 1 -- 172.21.15.52:0/1906961274 learned_addr learned my addr 172.21.15.52:0/1906961274 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc877fff640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8780d2320 0x7fc878146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87cc68640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8780cfa50 0x7fc87813c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87d469640 1 -- 172.21.15.52:0/1906961274 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8780cfa50 msgr2=0x7fc87813c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87d469640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8780cfa50 0x7fc87813c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87d469640 1 -- 172.21.15.52:0/1906961274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8780d2320 msgr2=0x7fc878146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87d469640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8780d2320 0x7fc878146ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87d469640 1 -- 172.21.15.52:0/1906961274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87815dae0 con 0x7fc87813d000 2024-05-04T21:20:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc877fff640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8780d2320 0x7fc878146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T21:20:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87d469640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87813d000 0x7fc878147430 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fc87000c700 tx=0x7fc87000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc8757fa640 1 -- 172.21.15.52:0/1906961274 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870018070 con 0x7fc87813d000 2024-05-04T21:20:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc8757fa640 1 -- 172.21.15.52:0/1906961274 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc87000ce10 con 0x7fc87813d000 2024-05-04T21:20:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 -- 172.21.15.52:0/1906961274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87815ddc0 con 0x7fc87813d000 2024-05-04T21:20:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc8757fa640 1 -- 172.21.15.52:0/1906961274 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87000a860 con 0x7fc87813d000 2024-05-04T21:20:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.843+0000 7fc87eecc640 1 -- 172.21.15.52:0/1906961274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87815e380 con 0x7fc87813d000 2024-05-04T21:20:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.847+0000 7fc87eecc640 1 -- 172.21.15.52:0/1906961274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc878062e50 con 0x7fc87813d000 2024-05-04T21:20:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.847+0000 7fc8757fa640 1 -- 172.21.15.52:0/1906961274 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc87001b070 con 0x7fc87813d000 2024-05-04T21:20:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.847+0000 7fc8757fa640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc85c0678f0 0x7fc85c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.847+0000 7fc8757fa640 1 -- 172.21.15.52:0/1906961274 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fc870084f20 con 0x7fc87813d000 2024-05-04T21:20:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.847+0000 7fc87cc68640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc85c0678f0 0x7fc85c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.847+0000 7fc87cc68640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc85c0678f0 0x7fc85c069db0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0ad000 tx=0x7fc86c0a4f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:31.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:31.851+0000 7fc8757fa640 1 -- 172.21.15.52:0/1906961274 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc870055d60 con 0x7fc87813d000 2024-05-04T21:20:32.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:32.035+0000 7fc87eecc640 1 -- 172.21.15.52:0/1906961274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc878075320 con 0x7fc87813d000 2024-05-04T21:20:32.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:32.035+0000 7fc8757fa640 1 -- 172.21.15.52:0/1906961274 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+13032 (secure 0 0 0) 0x7fc870055660 con 0x7fc87813d000 2024-05-04T21:20:32.060 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:32.060 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":37,"fsid":"708d9177-9f3e-4c4b-9d29-7b98768b2a25","created":"2024-05-04T12:17:57.932943+0000","modified":"2024-05-04T21:20:26.196381+0000","last_up_change":"2024-05-04T12:18:09.253892+0000","last_in_change":"2024-05-04T12:18:00.351464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T12:18:10.817553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b7012e43-90ab-42b8-b285-d5dc8e7f23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6809","nonce":2539299307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6811","nonce":2539299307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6815","nonce":2539299307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2539299307},{"type":"v1","addr":"172.21.15.52:6813","nonce":2539299307}]},"public_addr":"172.21.15.52:6809/2539299307","cluster_addr":"172.21.15.52:6811/2539299307","heartbeat_back_addr":"172.21.15.52:6815/2539299307","heartbeat_front_addr":"172.21.15.52:6813/2539299307","state":["exists","up"]},{"osd":1,"uuid":"1d96acd5-48ed-45ac-90df-dd9f646bcd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6817","nonce":1070276451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6819","nonce":1070276451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6823","nonce":1070276451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1070276451},{"type":"v1","addr":"172.21.15.52:6821","nonce":1070276451}]},"public_addr":"172.21.15.52:6817/1070276451","cluster_addr":"172.21.15.52:6819/1070276451","heartbeat_back_addr":"172.21.15.52:6823/1070276451","heartbeat_front_addr":"172.21.15.52:6821/1070276451","state":["exists","up"]},{"osd":2,"uuid":"6e8c3667-2a7d-4ee6-b5b7-81b6f2bed517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6801","nonce":4058473938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6803","nonce":4058473938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6807","nonce":4058473938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4058473938},{"type":"v1","addr":"172.21.15.52:6805","nonce":4058473938}]},"public_addr":"172.21.15.52:6801/4058473938","cluster_addr":"172.21.15.52:6803/4058473938","heartbeat_back_addr":"172.21.15.52:6807/4058473938","heartbeat_front_addr":"172.21.15.52:6805/4058473938","state":["exists","up"]},{"osd":3,"uuid":"bf071341-8de8-48e7-9454-c7301ca20351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6825","nonce":2130169459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6827","nonce":2130169459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6831","nonce":2130169459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2130169459},{"type":"v1","addr":"172.21.15.52:6829","nonce":2130169459}]},"public_addr":"172.21.15.52:6825/2130169459","cluster_addr":"172.21.15.52:6827/2130169459","heartbeat_back_addr":"172.21.15.52:6831/2130169459","heartbeat_front_addr":"172.21.15.52:6829/2130169459","state":["exists","up"]},{"osd":4,"uuid":"aa6cd584-fce4-4997-867d-587d65f3b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6825","nonce":2919151955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6827","nonce":2919151955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6831","nonce":2919151955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2919151955},{"type":"v1","addr":"172.21.15.111:6829","nonce":2919151955}]},"public_addr":"172.21.15.111:6825/2919151955","cluster_addr":"172.21.15.111:6827/2919151955","heartbeat_back_addr":"172.21.15.111:6831/2919151955","heartbeat_front_addr":"172.21.15.111:6829/2919151955","state":["exists","up"]},{"osd":5,"uuid":"7115a071-f397-4df5-b177-83637c288ff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6803","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6807","nonce":1318681540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6813","nonce":1318681540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6824","nonce":1318681540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1318681540},{"type":"v1","addr":"172.21.15.111:6819","nonce":1318681540}]},"public_addr":"172.21.15.111:6807/1318681540","cluster_addr":"172.21.15.111:6813/1318681540","heartbeat_back_addr":"172.21.15.111:6824/1318681540","heartbeat_front_addr":"172.21.15.111:6819/1318681540","state":["exists","up"]},{"osd":6,"uuid":"35f3218b-c24f-44b6-81b0-dafaa4dc084e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6801","nonce":1985934042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6805","nonce":1985934042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6818","nonce":1985934042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1985934042},{"type":"v1","addr":"172.21.15.111:6811","nonce":1985934042}]},"public_addr":"172.21.15.111:6801/1985934042","cluster_addr":"172.21.15.111:6805/1985934042","heartbeat_back_addr":"172.21.15.111:6818/1985934042","heartbeat_front_addr":"172.21.15.111:6811/1985934042","state":["exists","up"]},{"osd":7,"uuid":"1a440ec5-ccd5-49ea-a894-7a03a742640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6806","nonce":3267577455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6809","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6812","nonce":3267577455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6821","nonce":3267577455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":3267577455},{"type":"v1","addr":"172.21.15.111:6817","nonce":3267577455}]},"public_addr":"172.21.15.111:6806/3267577455","cluster_addr":"172.21.15.111:6812/3267577455","heartbeat_back_addr":"172.21.15.111:6821/3267577455","heartbeat_front_addr":"172.21.15.111:6817/3267577455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.810591+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.958451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.962966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.944474+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.879889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:06.015626+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.955736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T12:18:05.886612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6837/1620779861":"2024-05-05T21:20:23.113036+0000","172.21.15.52:6836/1620779861":"2024-05-05T21:20:23.113036+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T21:20:32.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:32.035+0000 7fc87eecc640 1 -- 172.21.15.52:0/1906961274 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc85c0678f0 msgr2=0x7fc85c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:32.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:32.035+0000 7fc87eecc640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc85c0678f0 0x7fc85c069db0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0ad000 tx=0x7fc86c0a4f90 comp rx=0 tx=0).stop 2024-05-04T21:20:32.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:32.035+0000 7fc87eecc640 1 -- 172.21.15.52:0/1906961274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87813d000 msgr2=0x7fc878147430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:32.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:32.035+0000 7fc87eecc640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87813d000 0x7fc878147430 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fc87000c700 tx=0x7fc87000cbc0 comp rx=0 tx=0).stop 2024-05-04T21:20:32.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:32.035+0000 7fc87eecc640 1 -- 172.21.15.52:0/1906961274 shutdown_connections 2024-05-04T21:20:32.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:32.035+0000 7fc87eecc640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc85c0678f0 0x7fc85c069db0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:32.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:32.035+0000 7fc87eecc640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87813d000 0x7fc878147430 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:32.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:32.035+0000 7fc87eecc640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8780d2320 0x7fc878146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:32.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:32.035+0000 7fc87eecc640 1 --2- 172.21.15.52:0/1906961274 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8780cfa50 0x7fc87813c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:32.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:32.035+0000 7fc87eecc640 1 -- 172.21.15.52:0/1906961274 >> 172.21.15.52:0/1906961274 conn(0x7fc87806b210 msgr2=0x7fc8780cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:32.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:32.035+0000 7fc87eecc640 1 -- 172.21.15.52:0/1906961274 shutdown_connections 2024-05-04T21:20:32.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:32.035+0000 7fc87eecc640 1 -- 172.21.15.52:0/1906961274 wait complete. 2024-05-04T21:20:33.064 INFO:tasks.ceph:Scrubbing osd.0 2024-05-04T21:20:33.064 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-04T21:20:33.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe621cbe640 1 Processor -- start 2024-05-04T21:20:33.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe621cbe640 1 -- start start 2024-05-04T21:20:33.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe621cbe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe61c1553d0 0x7fe61c0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe621cbe640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe61c0cfa70 0x7fe61c15c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe621cbe640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe61c0cdeb0 con 0x7fe61c15c150 2024-05-04T21:20:33.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe621cbe640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fe61c0cd2a0 con 0x7fe61c1553d0 2024-05-04T21:20:33.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe621cbe640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe61c0cd580 con 0x7fe61c0cfa70 2024-05-04T21:20:33.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe61b7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe61c1553d0 0x7fe61c0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe61b7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe61c1553d0 0x7fe61c0cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59786/0 (socket says 172.21.15.52:59786) 2024-05-04T21:20:33.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe61b7fe640 1 -- 172.21.15.52:0/3087342955 learned_addr learned my addr 172.21.15.52:0/3087342955 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:33.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe61bfff640 1 --2- 172.21.15.52:0/3087342955 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe61c0cfa70 0x7fe61c15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe61b7fe640 1 -- 172.21.15.52:0/3087342955 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe61c0cfa70 msgr2=0x7fe61c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:33.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe61b7fe640 1 --2- 172.21.15.52:0/3087342955 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe61c0cfa70 0x7fe61c15c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe61b7fe640 1 -- 172.21.15.52:0/3087342955 >> v1:172.21.15.111:6789/0 conn(0x7fe61c15c150 legacy=0x7fe61c0cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:33.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.199+0000 7fe61b7fe640 1 -- 172.21.15.52:0/3087342955 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c0cd860 con 0x7fe61c1553d0 2024-05-04T21:20:33.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.203+0000 7fe61b7fe640 1 --2- 172.21.15.52:0/3087342955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe61c1553d0 0x7fe61c0cc930 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fe61c070cf0 tx=0x7fe60c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51dbcaa3226e1d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:33.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.203+0000 7fe61a7fc640 1 -- 172.21.15.52:0/3087342955 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0b1070 con 0x7fe61c1553d0 2024-05-04T21:20:33.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.203+0000 7fe61a7fc640 1 -- 172.21.15.52:0/3087342955 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe60c0ac600 con 0x7fe61c1553d0 2024-05-04T21:20:33.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.203+0000 7fe61a7fc640 1 -- 172.21.15.52:0/3087342955 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0ac9d0 con 0x7fe61c1553d0 2024-05-04T21:20:33.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.203+0000 7fe621cbe640 1 -- 172.21.15.52:0/3087342955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe61c1553d0 msgr2=0x7fe61c0cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.203+0000 7fe621cbe640 1 --2- 172.21.15.52:0/3087342955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe61c1553d0 0x7fe61c0cc930 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fe61c070cf0 tx=0x7fe60c01c4d0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.203+0000 7fe621cbe640 1 -- 172.21.15.52:0/3087342955 shutdown_connections 2024-05-04T21:20:33.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.203+0000 7fe621cbe640 1 --2- 172.21.15.52:0/3087342955 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe61c0cfa70 0x7fe61c15c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.203+0000 7fe621cbe640 1 --2- 172.21.15.52:0/3087342955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe61c1553d0 0x7fe61c0cc930 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.203+0000 7fe621cbe640 1 -- 172.21.15.52:0/3087342955 >> 172.21.15.52:0/3087342955 conn(0x7fe61c067760 msgr2=0x7fe61c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:33.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.203+0000 7fe621cbe640 1 -- 172.21.15.52:0/3087342955 shutdown_connections 2024-05-04T21:20:33.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.203+0000 7fe621cbe640 1 -- 172.21.15.52:0/3087342955 wait complete. 2024-05-04T21:20:33.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe621cbe640 1 Processor -- start 2024-05-04T21:20:33.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe621cbe640 1 -- start start 2024-05-04T21:20:33.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe621cbe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe61c0cfa70 0x7fe61c14ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe621cbe640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe61c1553d0 0x7fe61c13c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe621cbe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe61c15c150 0x7fe61c13ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe621cbe640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c15d5d0 con 0x7fe61c15c150 2024-05-04T21:20:33.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe621cbe640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c0cd080 con 0x7fe61c0cfa70 2024-05-04T21:20:33.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe621cbe640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fe61c15d750 con 0x7fe61c1553d0 2024-05-04T21:20:33.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe61bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe61c15c150 0x7fe61c13ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe61bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe61c15c150 0x7fe61c13ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59788/0 (socket says 172.21.15.52:59788) 2024-05-04T21:20:33.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe61bfff640 1 -- 172.21.15.52:0/3614832915 learned_addr learned my addr 172.21.15.52:0/3614832915 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe61bfff640 1 -- 172.21.15.52:0/3614832915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe61c1553d0 msgr2=0x7fe61c13c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe61bfff640 1 --2- 172.21.15.52:0/3614832915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe61c1553d0 0x7fe61c13c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe61bfff640 1 -- 172.21.15.52:0/3614832915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe61c0cfa70 msgr2=0x7fe61c14ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe61b7fe640 1 --2- 172.21.15.52:0/3614832915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe61c0cfa70 0x7fe61c14ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe61bfff640 1 --2- 172.21.15.52:0/3614832915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe61c0cfa70 0x7fe61c14ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe61bfff640 1 -- 172.21.15.52:0/3614832915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c14d470 con 0x7fe61c15c150 2024-05-04T21:20:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe61bfff640 1 --2- 172.21.15.52:0/3614832915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe61c15c150 0x7fe61c13ca30 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fe61400cc30 tx=0x7fe61400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe603fff640 1 -- 172.21.15.52:0/3614832915 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61400ce40 con 0x7fe61c15c150 2024-05-04T21:20:33.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe603fff640 1 -- 172.21.15.52:0/3614832915 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6140039a0 con 0x7fe61c15c150 2024-05-04T21:20:33.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe621cbe640 1 -- 172.21.15.52:0/3614832915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61c14d750 con 0x7fe61c15c150 2024-05-04T21:20:33.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe603fff640 1 -- 172.21.15.52:0/3614832915 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614003d50 con 0x7fe61c15c150 2024-05-04T21:20:33.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe621cbe640 1 -- 172.21.15.52:0/3614832915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61c14da30 con 0x7fe61c15c150 2024-05-04T21:20:33.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.207+0000 7fe621cbe640 1 -- 172.21.15.52:0/3614832915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe61c152fc0 con 0x7fe61c15c150 2024-05-04T21:20:33.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.211+0000 7fe603fff640 1 -- 172.21.15.52:0/3614832915 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe614003ef0 con 0x7fe61c15c150 2024-05-04T21:20:33.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.211+0000 7fe603fff640 1 --2- 172.21.15.52:0/3614832915 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe608067940 0x7fe608069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.211+0000 7fe603fff640 1 -- 172.21.15.52:0/3614832915 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fe6140863c0 con 0x7fe61c15c150 2024-05-04T21:20:33.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.211+0000 7fe61b7fe640 1 --2- 172.21.15.52:0/3614832915 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe608067940 0x7fe608069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.211+0000 7fe603fff640 1 --2- 172.21.15.52:0/3614832915 >> [v2:172.21.15.52:6808/2539299307,v1:172.21.15.52:6809/2539299307] conn(0x7fe60806d800 0x7fe60806fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.211+0000 7fe603fff640 1 -- 172.21.15.52:0/3614832915 --> [v2:172.21.15.52:6808/2539299307,v1:172.21.15.52:6809/2539299307] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe61401ae50 con 0x7fe60806d800 2024-05-04T21:20:33.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.211+0000 7fe603fff640 1 -- 172.21.15.52:0/3614832915 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe614086770 con 0x7fe61c15c150 2024-05-04T21:20:33.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.211+0000 7fe61affd640 1 --2- 172.21.15.52:0/3614832915 >> [v2:172.21.15.52:6808/2539299307,v1:172.21.15.52:6809/2539299307] conn(0x7fe60806d800 0x7fe60806fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.211+0000 7fe61b7fe640 1 --2- 172.21.15.52:0/3614832915 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe608067940 0x7fe608069e00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe60c0ae000 tx=0x7fe60c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:33.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.211+0000 7fe61affd640 1 --2- 172.21.15.52:0/3614832915 >> [v2:172.21.15.52:6808/2539299307,v1:172.21.15.52:6809/2539299307] conn(0x7fe60806d800 0x7fe60806fc20 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:33.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.327+0000 7fe603fff640 1 -- 172.21.15.52:0/3614832915 <== osd.0 v2:172.21.15.52:6808/2539299307 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe61401ae50 con 0x7fe60806d800 2024-05-04T21:20:33.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.343+0000 7fe621cbe640 1 -- 172.21.15.52:0/3614832915 --> [v2:172.21.15.52:6808/2539299307,v1:172.21.15.52:6809/2539299307] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe61c062e50 con 0x7fe60806d800 2024-05-04T21:20:33.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.379+0000 7fe603fff640 1 -- 172.21.15.52:0/3614832915 <== osd.0 v2:172.21.15.52:6808/2539299307 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe61c062e50 con 0x7fe60806d800 2024-05-04T21:20:33.386 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-04T21:20:33.386 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-04T21:20:33.386 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-04T21:20:33.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.383+0000 7fe621cbe640 1 -- 172.21.15.52:0/3614832915 >> [v2:172.21.15.52:6808/2539299307,v1:172.21.15.52:6809/2539299307] conn(0x7fe60806d800 msgr2=0x7fe60806fc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.383+0000 7fe621cbe640 1 --2- 172.21.15.52:0/3614832915 >> [v2:172.21.15.52:6808/2539299307,v1:172.21.15.52:6809/2539299307] conn(0x7fe60806d800 0x7fe60806fc20 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.383+0000 7fe621cbe640 1 -- 172.21.15.52:0/3614832915 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe608067940 msgr2=0x7fe608069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.383+0000 7fe621cbe640 1 --2- 172.21.15.52:0/3614832915 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fe608067940 0x7fe608069e00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe60c0ae000 tx=0x7fe60c008040 comp rx=0 tx=0).stop 2024-05-04T21:20:33.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.383+0000 7fe621cbe640 1 -- 172.21.15.52:0/3614832915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe61c15c150 msgr2=0x7fe61c13ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.383+0000 7fe621cbe640 1 --2- 172.21.15.52:0/3614832915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe61c15c150 0x7fe61c13ca30 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fe61400cc30 tx=0x7fe61400a5c0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.383+0000 7fe61bfff640 1 -- 172.21.15.52:0/3614832915 reap_dead start 2024-05-04T21:20:33.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.383+0000 7fe621cbe640 1 -- 172.21.15.52:0/3614832915 shutdown_connections 2024-05-04T21:20:33.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.383+0000 7fe621cbe640 1 -- 172.21.15.52:0/3614832915 >> 172.21.15.52:0/3614832915 conn(0x7fe61c067760 msgr2=0x7fe61c15e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:33.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.383+0000 7fe621cbe640 1 -- 172.21.15.52:0/3614832915 shutdown_connections 2024-05-04T21:20:33.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.383+0000 7fe621cbe640 1 -- 172.21.15.52:0/3614832915 wait complete. 2024-05-04T21:20:33.410 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-04T21:20:33.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681c3b0640 1 Processor -- start 2024-05-04T21:20:33.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681c3b0640 1 -- start start 2024-05-04T21:20:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681c3b0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f68140ccdb0 0x7f68140723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681c3b0640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f681415c7f0 0x7f681415c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681c3b0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f68140cd190 0x7f6814158640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681c3b0640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f6814070690 con 0x7f68140ccdb0 2024-05-04T21:20:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681c3b0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f6814072aa0 con 0x7f68140cd190 2024-05-04T21:20:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681c3b0640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f68140708e0 con 0x7f681415c7f0 2024-05-04T21:20:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681a14c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f68140ccdb0 0x7f68140723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681a14c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f68140ccdb0 0x7f68140723e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59792/0 (socket says 172.21.15.52:59792) 2024-05-04T21:20:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681a14c640 1 -- 172.21.15.52:0/4213784503 learned_addr learned my addr 172.21.15.52:0/4213784503 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681994b640 1 --2- 172.21.15.52:0/4213784503 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f681415c7f0 0x7f681415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681a94d640 1 --2- 172.21.15.52:0/4213784503 >> v2:172.21.15.111:3300/0 conn(0x7f68140cd190 0x7f6814158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681a14c640 1 -- 172.21.15.52:0/4213784503 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f681415c7f0 msgr2=0x7f681415c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:33.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681a14c640 1 --2- 172.21.15.52:0/4213784503 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f681415c7f0 0x7f681415c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681a14c640 1 -- 172.21.15.52:0/4213784503 >> v2:172.21.15.111:3300/0 conn(0x7f68140cd190 msgr2=0x7f6814158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:33.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681a14c640 1 --2- 172.21.15.52:0/4213784503 >> v2:172.21.15.111:3300/0 conn(0x7f68140cd190 0x7f6814158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681a14c640 1 -- 172.21.15.52:0/4213784503 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6814070bc0 con 0x7f68140ccdb0 2024-05-04T21:20:33.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.535+0000 7f681994b640 1 --2- 172.21.15.52:0/4213784503 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f681415c7f0 0x7f681415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T21:20:33.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.539+0000 7f681a14c640 1 --2- 172.21.15.52:0/4213784503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68140ccdb0 0x7f68140723e0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f6814072d80 tx=0x7f681001c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf195df89896da4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:33.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.539+0000 7f681914a640 1 -- 172.21.15.52:0/4213784503 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68100b0070 con 0x7f68140ccdb0 2024-05-04T21:20:33.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.539+0000 7f681914a640 1 -- 172.21.15.52:0/4213784503 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68100ab600 con 0x7f68140ccdb0 2024-05-04T21:20:33.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.539+0000 7f681914a640 1 -- 172.21.15.52:0/4213784503 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68100ab9d0 con 0x7f68140ccdb0 2024-05-04T21:20:33.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.539+0000 7f681c3b0640 1 -- 172.21.15.52:0/4213784503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68140ccdb0 msgr2=0x7f68140723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.539+0000 7f681c3b0640 1 --2- 172.21.15.52:0/4213784503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68140ccdb0 0x7f68140723e0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f6814072d80 tx=0x7f681001c1d0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.539+0000 7f681c3b0640 1 -- 172.21.15.52:0/4213784503 shutdown_connections 2024-05-04T21:20:33.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.539+0000 7f681c3b0640 1 --2- 172.21.15.52:0/4213784503 >> v2:172.21.15.111:3300/0 conn(0x7f68140cd190 0x7f6814158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.539+0000 7f681c3b0640 1 --2- 172.21.15.52:0/4213784503 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f681415c7f0 0x7f681415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.539+0000 7f681c3b0640 1 --2- 172.21.15.52:0/4213784503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68140ccdb0 0x7f68140723e0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.539+0000 7f681c3b0640 1 -- 172.21.15.52:0/4213784503 >> 172.21.15.52:0/4213784503 conn(0x7f68140b14a0 msgr2=0x7f68140b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:33.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.539+0000 7f681c3b0640 1 -- 172.21.15.52:0/4213784503 shutdown_connections 2024-05-04T21:20:33.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.539+0000 7f681c3b0640 1 -- 172.21.15.52:0/4213784503 wait complete. 2024-05-04T21:20:33.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681c3b0640 1 Processor -- start 2024-05-04T21:20:33.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681c3b0640 1 -- start start 2024-05-04T21:20:33.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681c3b0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68140ccdb0 0x7f6814151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681c3b0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68140cd190 0x7f6814163c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681c3b0640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f681415c7f0 0x7f68140ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681c3b0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6814071f40 con 0x7f68140cd190 2024-05-04T21:20:33.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681c3b0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6814071dc0 con 0x7f68140ccdb0 2024-05-04T21:20:33.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681c3b0640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f68140720c0 con 0x7f681415c7f0 2024-05-04T21:20:33.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681994b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68140cd190 0x7f6814163c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681994b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68140cd190 0x7f6814163c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59804/0 (socket says 172.21.15.52:59804) 2024-05-04T21:20:33.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681994b640 1 -- 172.21.15.52:0/1681344393 learned_addr learned my addr 172.21.15.52:0/1681344393 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:33.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681a14c640 1 --2- 172.21.15.52:0/1681344393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68140ccdb0 0x7f6814151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681a94d640 1 --2- 172.21.15.52:0/1681344393 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f681415c7f0 0x7f68140ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681994b640 1 -- 172.21.15.52:0/1681344393 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f681415c7f0 msgr2=0x7f68140ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681994b640 1 --2- 172.21.15.52:0/1681344393 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f681415c7f0 0x7f68140ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681994b640 1 -- 172.21.15.52:0/1681344393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68140ccdb0 msgr2=0x7f6814151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681994b640 1 --2- 172.21.15.52:0/1681344393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68140ccdb0 0x7f6814151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681994b640 1 -- 172.21.15.52:0/1681344393 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6814164630 con 0x7f68140cd190 2024-05-04T21:20:33.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681994b640 1 --2- 172.21.15.52:0/1681344393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68140cd190 0x7f6814163c30 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f680400d830 tx=0x7f680400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:33.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f680affd640 1 -- 172.21.15.52:0/1681344393 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804018020 con 0x7f68140cd190 2024-05-04T21:20:33.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f680affd640 1 -- 172.21.15.52:0/1681344393 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68040096c0 con 0x7f68140cd190 2024-05-04T21:20:33.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f680affd640 1 -- 172.21.15.52:0/1681344393 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804009ab0 con 0x7f68140cd190 2024-05-04T21:20:33.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681c3b0640 1 -- 172.21.15.52:0/1681344393 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6814164910 con 0x7f68140cd190 2024-05-04T21:20:33.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681c3b0640 1 -- 172.21.15.52:0/1681344393 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6814164ed0 con 0x7f68140cd190 2024-05-04T21:20:33.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f681c3b0640 1 -- 172.21.15.52:0/1681344393 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6814062e50 con 0x7f68140cd190 2024-05-04T21:20:33.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f680affd640 1 -- 172.21.15.52:0/1681344393 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f680401b020 con 0x7f68140cd190 2024-05-04T21:20:33.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.543+0000 7f680affd640 1 --2- 172.21.15.52:0/1681344393 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67f80678c0 0x7f67f8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.547+0000 7f680affd640 1 -- 172.21.15.52:0/1681344393 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f6804086160 con 0x7f68140cd190 2024-05-04T21:20:33.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.547+0000 7f681a14c640 1 --2- 172.21.15.52:0/1681344393 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67f80678c0 0x7f67f8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.547+0000 7f681a14c640 1 --2- 172.21.15.52:0/1681344393 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67f80678c0 0x7f67f8069d80 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f68100ad000 tx=0x7f6810008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:33.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.551+0000 7f680affd640 1 -- 172.21.15.52:0/1681344393 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6804022020 con 0x7f68140cd190 2024-05-04T21:20:33.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.715+0000 7f681c3b0640 1 -- 172.21.15.52:0/1681344393 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f6814075320 con 0x7f67f80678c0 2024-05-04T21:20:33.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.727+0000 7f680affd640 1 -- 172.21.15.52:0/1681344393 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6814075320 con 0x7f67f80678c0 2024-05-04T21:20:33.732 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 0 to deep-scrub 2024-05-04T21:20:33.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.731+0000 7f681c3b0640 1 -- 172.21.15.52:0/1681344393 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67f80678c0 msgr2=0x7f67f8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.731+0000 7f681c3b0640 1 --2- 172.21.15.52:0/1681344393 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67f80678c0 0x7f67f8069d80 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f68100ad000 tx=0x7f6810008040 comp rx=0 tx=0).stop 2024-05-04T21:20:33.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.731+0000 7f681c3b0640 1 -- 172.21.15.52:0/1681344393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68140cd190 msgr2=0x7f6814163c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.731+0000 7f681c3b0640 1 --2- 172.21.15.52:0/1681344393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68140cd190 0x7f6814163c30 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f680400d830 tx=0x7f680400dcf0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.731+0000 7f681c3b0640 1 -- 172.21.15.52:0/1681344393 shutdown_connections 2024-05-04T21:20:33.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.731+0000 7f681c3b0640 1 --2- 172.21.15.52:0/1681344393 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f67f80678c0 0x7f67f8069d80 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.731+0000 7f681c3b0640 1 --2- 172.21.15.52:0/1681344393 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f681415c7f0 0x7f68140ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.731+0000 7f681c3b0640 1 --2- 172.21.15.52:0/1681344393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68140cd190 0x7f6814163c30 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.731+0000 7f681c3b0640 1 --2- 172.21.15.52:0/1681344393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68140ccdb0 0x7f6814151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.731+0000 7f681c3b0640 1 -- 172.21.15.52:0/1681344393 >> 172.21.15.52:0/1681344393 conn(0x7f68140b14a0 msgr2=0x7f681415de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:33.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.731+0000 7f681c3b0640 1 -- 172.21.15.52:0/1681344393 shutdown_connections 2024-05-04T21:20:33.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.731+0000 7f681c3b0640 1 -- 172.21.15.52:0/1681344393 wait complete. 2024-05-04T21:20:33.752 INFO:tasks.ceph:Scrubbing osd.1 2024-05-04T21:20:33.752 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-04T21:20:33.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d85e5640 1 Processor -- start 2024-05-04T21:20:33.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d85e5640 1 -- start start 2024-05-04T21:20:33.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d85e5640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd0d01608f0 0x7fd0d006ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d85e5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd0d006f390 0x7fd0d00ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d85e5640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0d00ced50 con 0x7fd0d0144fc0 2024-05-04T21:20:33.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d85e5640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fd0d01453a0 con 0x7fd0d01608f0 2024-05-04T21:20:33.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d85e5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fd0d015fbc0 con 0x7fd0d006f390 2024-05-04T21:20:33.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d5b80640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd0d01608f0 0x7fd0d006ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d6381640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fd0d0144fc0 0x7fd0d00ce300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55466/0 (socket says 172.21.15.52:55466) 2024-05-04T21:20:33.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d5b80640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd0d01608f0 0x7fd0d006ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59806/0 (socket says 172.21.15.52:59806) 2024-05-04T21:20:33.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d6381640 1 -- 172.21.15.52:0/2576243000 learned_addr learned my addr 172.21.15.52:0/2576243000 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:33.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d6b82640 1 --2- 172.21.15.52:0/2576243000 >> v2:172.21.15.111:3300/0 conn(0x7fd0d006f390 0x7fd0d00ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d5b80640 1 -- 172.21.15.52:0/2576243000 >> v2:172.21.15.111:3300/0 conn(0x7fd0d006f390 msgr2=0x7fd0d00ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:33.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d5b80640 1 --2- 172.21.15.52:0/2576243000 >> v2:172.21.15.111:3300/0 conn(0x7fd0d006f390 0x7fd0d00ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d5b80640 1 -- 172.21.15.52:0/2576243000 >> v1:172.21.15.52:6789/0 conn(0x7fd0d0144fc0 legacy=0x7fd0d00ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.879+0000 7fd0d5b80640 1 -- 172.21.15.52:0/2576243000 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d015fe10 con 0x7fd0d01608f0 2024-05-04T21:20:33.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.883+0000 7fd0d5b80640 1 --2- 172.21.15.52:0/2576243000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d01608f0 0x7fd0d006ee50 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fd0d006fef0 tx=0x7fd0c001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fc6e45c566c20d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:33.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.883+0000 7fd0d537f640 1 -- 172.21.15.52:0/2576243000 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00b0070 con 0x7fd0d01608f0 2024-05-04T21:20:33.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.883+0000 7fd0d537f640 1 -- 172.21.15.52:0/2576243000 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c00ab600 con 0x7fd0d01608f0 2024-05-04T21:20:33.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.883+0000 7fd0d537f640 1 -- 172.21.15.52:0/2576243000 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00ab9d0 con 0x7fd0d01608f0 2024-05-04T21:20:33.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.883+0000 7fd0d85e5640 1 -- 172.21.15.52:0/2576243000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d01608f0 msgr2=0x7fd0d006ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.883+0000 7fd0d85e5640 1 --2- 172.21.15.52:0/2576243000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d01608f0 0x7fd0d006ee50 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fd0d006fef0 tx=0x7fd0c001c3f0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.883+0000 7fd0d85e5640 1 -- 172.21.15.52:0/2576243000 shutdown_connections 2024-05-04T21:20:33.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.883+0000 7fd0d85e5640 1 --2- 172.21.15.52:0/2576243000 >> v2:172.21.15.111:3300/0 conn(0x7fd0d006f390 0x7fd0d00ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.883+0000 7fd0d85e5640 1 --2- 172.21.15.52:0/2576243000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d01608f0 0x7fd0d006ee50 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.883+0000 7fd0d85e5640 1 -- 172.21.15.52:0/2576243000 >> 172.21.15.52:0/2576243000 conn(0x7fd0d0067760 msgr2=0x7fd0d00b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:33.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.883+0000 7fd0d85e5640 1 -- 172.21.15.52:0/2576243000 shutdown_connections 2024-05-04T21:20:33.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.883+0000 7fd0d85e5640 1 -- 172.21.15.52:0/2576243000 wait complete. 2024-05-04T21:20:33.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d85e5640 1 Processor -- start 2024-05-04T21:20:33.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d85e5640 1 -- start start 2024-05-04T21:20:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d85e5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0d006f390 0x7fd0d01386a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d85e5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0144fc0 0x7fd0d0159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d85e5640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0d01608f0 0x7fd0d0159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d85e5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d0072250 con 0x7fd0d0144fc0 2024-05-04T21:20:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d85e5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d00720d0 con 0x7fd0d006f390 2024-05-04T21:20:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d85e5640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d0166f50 con 0x7fd0d01608f0 2024-05-04T21:20:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d5b80640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0144fc0 0x7fd0d0159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d5b80640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0144fc0 0x7fd0d0159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59808/0 (socket says 172.21.15.52:59808) 2024-05-04T21:20:33.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d5b80640 1 -- 172.21.15.52:0/3867366839 learned_addr learned my addr 172.21.15.52:0/3867366839 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:33.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d6381640 1 --2- 172.21.15.52:0/3867366839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0d006f390 0x7fd0d01386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d6b82640 1 --2- 172.21.15.52:0/3867366839 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0d01608f0 0x7fd0d0159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d5b80640 1 -- 172.21.15.52:0/3867366839 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0d01608f0 msgr2=0x7fd0d0159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d5b80640 1 --2- 172.21.15.52:0/3867366839 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0d01608f0 0x7fd0d0159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d5b80640 1 -- 172.21.15.52:0/3867366839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0d006f390 msgr2=0x7fd0d01386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d5b80640 1 --2- 172.21.15.52:0/3867366839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0d006f390 0x7fd0d01386a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d5b80640 1 -- 172.21.15.52:0/3867366839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d0143a90 con 0x7fd0d0144fc0 2024-05-04T21:20:33.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d5b80640 1 --2- 172.21.15.52:0/3867366839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0144fc0 0x7fd0d0159450 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00ae040 tx=0x7fd0c0019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:33.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3867366839 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00b8020 con 0x7fd0d0144fc0 2024-05-04T21:20:33.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3867366839 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c00b4410 con 0x7fd0d0144fc0 2024-05-04T21:20:33.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d85e5640 1 -- 172.21.15.52:0/3867366839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d0142fd0 con 0x7fd0d0144fc0 2024-05-04T21:20:33.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3867366839 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00b0040 con 0x7fd0d0144fc0 2024-05-04T21:20:33.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d85e5640 1 -- 172.21.15.52:0/3867366839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d0144710 con 0x7fd0d0144fc0 2024-05-04T21:20:33.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d85e5640 1 -- 172.21.15.52:0/3867366839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd0d01441e0 con 0x7fd0d0144fc0 2024-05-04T21:20:33.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3867366839 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd0c00bb020 con 0x7fd0d0144fc0 2024-05-04T21:20:33.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0c6ffd640 1 --2- 172.21.15.52:0/3867366839 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd0b40678c0 0x7fd0b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0d6381640 1 --2- 172.21.15.52:0/3867366839 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd0b40678c0 0x7fd0b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.887+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3867366839 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fd0c011f9a0 con 0x7fd0d0144fc0 2024-05-04T21:20:33.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.891+0000 7fd0c6ffd640 1 --2- 172.21.15.52:0/3867366839 >> [v2:172.21.15.52:6816/1070276451,v1:172.21.15.52:6817/1070276451] conn(0x7fd0b406d780 0x7fd0b406fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:33.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.891+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3867366839 --> [v2:172.21.15.52:6816/1070276451,v1:172.21.15.52:6817/1070276451] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd0c00ace50 con 0x7fd0b406d780 2024-05-04T21:20:33.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.891+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3867366839 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd0c011fe30 con 0x7fd0d0144fc0 2024-05-04T21:20:33.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.891+0000 7fd0d6b82640 1 --2- 172.21.15.52:0/3867366839 >> [v2:172.21.15.52:6816/1070276451,v1:172.21.15.52:6817/1070276451] conn(0x7fd0b406d780 0x7fd0b406fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:33.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.891+0000 7fd0d6381640 1 --2- 172.21.15.52:0/3867366839 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd0b40678c0 0x7fd0b4069d80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc000f70 tx=0x7fd0cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:33.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.891+0000 7fd0d6b82640 1 --2- 172.21.15.52:0/3867366839 >> [v2:172.21.15.52:6816/1070276451,v1:172.21.15.52:6817/1070276451] conn(0x7fd0b406d780 0x7fd0b406fba0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:33.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.943+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3867366839 <== osd.1 v2:172.21.15.52:6816/1070276451 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd0c00ace50 con 0x7fd0b406d780 2024-05-04T21:20:33.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.959+0000 7fd0d85e5640 1 -- 172.21.15.52:0/3867366839 --> [v2:172.21.15.52:6816/1070276451,v1:172.21.15.52:6817/1070276451] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd0d0062e50 con 0x7fd0b406d780 2024-05-04T21:20:33.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.975+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3867366839 <== osd.1 v2:172.21.15.52:6816/1070276451 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd0d0062e50 con 0x7fd0b406d780 2024-05-04T21:20:33.980 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-04T21:20:33.980 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-04T21:20:33.980 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-04T21:20:33.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.975+0000 7fd0d85e5640 1 -- 172.21.15.52:0/3867366839 >> [v2:172.21.15.52:6816/1070276451,v1:172.21.15.52:6817/1070276451] conn(0x7fd0b406d780 msgr2=0x7fd0b406fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.975+0000 7fd0d85e5640 1 --2- 172.21.15.52:0/3867366839 >> [v2:172.21.15.52:6816/1070276451,v1:172.21.15.52:6817/1070276451] conn(0x7fd0b406d780 0x7fd0b406fba0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.975+0000 7fd0d85e5640 1 -- 172.21.15.52:0/3867366839 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd0b40678c0 msgr2=0x7fd0b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.975+0000 7fd0d85e5640 1 --2- 172.21.15.52:0/3867366839 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fd0b40678c0 0x7fd0b4069d80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc000f70 tx=0x7fd0cc009040 comp rx=0 tx=0).stop 2024-05-04T21:20:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.975+0000 7fd0d85e5640 1 -- 172.21.15.52:0/3867366839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0144fc0 msgr2=0x7fd0d0159450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.975+0000 7fd0d85e5640 1 --2- 172.21.15.52:0/3867366839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0144fc0 0x7fd0d0159450 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00ae040 tx=0x7fd0c0019190 comp rx=0 tx=0).stop 2024-05-04T21:20:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.975+0000 7fd0d6b82640 1 -- 172.21.15.52:0/3867366839 reap_dead start 2024-05-04T21:20:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.975+0000 7fd0d85e5640 1 -- 172.21.15.52:0/3867366839 shutdown_connections 2024-05-04T21:20:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.975+0000 7fd0d85e5640 1 -- 172.21.15.52:0/3867366839 >> 172.21.15.52:0/3867366839 conn(0x7fd0d0067760 msgr2=0x7fd0d00b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:33.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.975+0000 7fd0d85e5640 1 -- 172.21.15.52:0/3867366839 shutdown_connections 2024-05-04T21:20:33.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:33.975+0000 7fd0d85e5640 1 -- 172.21.15.52:0/3867366839 wait complete. 2024-05-04T21:20:34.004 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-04T21:20:34.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.127+0000 7fcf2bfa4640 1 Processor -- start 2024-05-04T21:20:34.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.127+0000 7fcf2bfa4640 1 -- start start 2024-05-04T21:20:34.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.127+0000 7fcf2bfa4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fcf2415c520 0x7fcf240ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.127+0000 7fcf2bfa4640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf240cd060 0x7fcf240ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.127+0000 7fcf2bfa4640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf24072f30 con 0x7fcf240cf430 2024-05-04T21:20:34.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.127+0000 7fcf2bfa4640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fcf240d0240 con 0x7fcf2415c520 2024-05-04T21:20:34.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.127+0000 7fcf2bfa4640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fcf240d0520 con 0x7fcf240cd060 2024-05-04T21:20:34.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.127+0000 7fcf29d40640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fcf2415c520 0x7fcf240ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.127+0000 7fcf2a541640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fcf240cf430 0x7fcf2415ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55470/0 (socket says 172.21.15.52:55470) 2024-05-04T21:20:34.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.127+0000 7fcf2a541640 1 -- 172.21.15.52:0/1053927284 learned_addr learned my addr 172.21.15.52:0/1053927284 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:34.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.127+0000 7fcf2953f640 1 --2- 172.21.15.52:0/1053927284 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf240cd060 0x7fcf240ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf29d40640 1 -- 172.21.15.52:0/1053927284 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf240cd060 msgr2=0x7fcf240ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:34.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf29d40640 1 --2- 172.21.15.52:0/1053927284 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf240cd060 0x7fcf240ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf29d40640 1 -- 172.21.15.52:0/1053927284 >> v1:172.21.15.52:6789/0 conn(0x7fcf240cf430 legacy=0x7fcf2415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf28d3e640 1 -- 172.21.15.52:0/1053927284 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 229874176 0 0) 0x7fcf24072f30 con 0x7fcf240cf430 2024-05-04T21:20:34.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf29d40640 1 -- 172.21.15.52:0/1053927284 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf240cb4b0 con 0x7fcf2415c520 2024-05-04T21:20:34.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf29d40640 1 --2- 172.21.15.52:0/1053927284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf2415c520 0x7fcf240ce9b0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fcf200097c0 tx=0x7fcf2001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ce5950595ea88 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:34.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf28d3e640 1 -- 172.21.15.52:0/1053927284 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf200b0070 con 0x7fcf2415c520 2024-05-04T21:20:34.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf28d3e640 1 -- 172.21.15.52:0/1053927284 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf200ab600 con 0x7fcf2415c520 2024-05-04T21:20:34.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf28d3e640 1 -- 172.21.15.52:0/1053927284 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf200ab9d0 con 0x7fcf2415c520 2024-05-04T21:20:34.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/1053927284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf2415c520 msgr2=0x7fcf240ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf2bfa4640 1 --2- 172.21.15.52:0/1053927284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf2415c520 0x7fcf240ce9b0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fcf200097c0 tx=0x7fcf2001c3e0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/1053927284 shutdown_connections 2024-05-04T21:20:34.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf2bfa4640 1 --2- 172.21.15.52:0/1053927284 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf240cd060 0x7fcf240ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf2bfa4640 1 --2- 172.21.15.52:0/1053927284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf2415c520 0x7fcf240ce9b0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/1053927284 >> 172.21.15.52:0/1053927284 conn(0x7fcf240cd5b0 msgr2=0x7fcf240cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:34.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/1053927284 shutdown_connections 2024-05-04T21:20:34.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.131+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/1053927284 wait complete. 2024-05-04T21:20:34.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf2bfa4640 1 Processor -- start 2024-05-04T21:20:34.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf2bfa4640 1 -- start start 2024-05-04T21:20:34.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf2bfa4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf240cd060 0x7fcf241484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf2bfa4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf240cf430 0x7fcf241469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf2bfa4640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf2415c520 0x7fcf24150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf2bfa4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcf24072230 con 0x7fcf240cd060 2024-05-04T21:20:34.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf2bfa4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcf240720b0 con 0x7fcf240cf430 2024-05-04T21:20:34.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf2bfa4640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fcf241692c0 con 0x7fcf2415c520 2024-05-04T21:20:34.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf29d40640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf240cd060 0x7fcf241484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf29d40640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf240cd060 0x7fcf241484a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59826/0 (socket says 172.21.15.52:59826) 2024-05-04T21:20:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf29d40640 1 -- 172.21.15.52:0/3667298707 learned_addr learned my addr 172.21.15.52:0/3667298707 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf2953f640 1 --2- 172.21.15.52:0/3667298707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf240cf430 0x7fcf241469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf2a541640 1 --2- 172.21.15.52:0/3667298707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf2415c520 0x7fcf24150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf29d40640 1 -- 172.21.15.52:0/3667298707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf2415c520 msgr2=0x7fcf24150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf29d40640 1 --2- 172.21.15.52:0/3667298707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf2415c520 0x7fcf24150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf29d40640 1 -- 172.21.15.52:0/3667298707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf240cf430 msgr2=0x7fcf241469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf29d40640 1 --2- 172.21.15.52:0/3667298707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf240cf430 0x7fcf241469a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf29d40640 1 -- 172.21.15.52:0/3667298707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf24147ee0 con 0x7fcf240cd060 2024-05-04T21:20:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf29d40640 1 --2- 172.21.15.52:0/3667298707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf240cd060 0x7fcf241484a0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fcf200ae040 tx=0x7fcf2001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf1a7fc640 1 -- 172.21.15.52:0/3667298707 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf200b0050 con 0x7fcf240cd060 2024-05-04T21:20:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf1a7fc640 1 -- 172.21.15.52:0/3667298707 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf200bc070 con 0x7fcf240cd060 2024-05-04T21:20:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/3667298707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf241481c0 con 0x7fcf240cd060 2024-05-04T21:20:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf1a7fc640 1 -- 172.21.15.52:0/3667298707 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf200b5870 con 0x7fcf240cd060 2024-05-04T21:20:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/3667298707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2415cd80 con 0x7fcf240cd060 2024-05-04T21:20:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/3667298707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf24062e50 con 0x7fcf240cd060 2024-05-04T21:20:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.135+0000 7fcf1a7fc640 1 -- 172.21.15.52:0/3667298707 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcf200b5a10 con 0x7fcf240cd060 2024-05-04T21:20:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.139+0000 7fcf1a7fc640 1 --2- 172.21.15.52:0/3667298707 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcf080678f0 0x7fcf08069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.139+0000 7fcf1a7fc640 1 -- 172.21.15.52:0/3667298707 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fcf20120160 con 0x7fcf240cd060 2024-05-04T21:20:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.139+0000 7fcf2953f640 1 --2- 172.21.15.52:0/3667298707 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcf080678f0 0x7fcf08069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.139+0000 7fcf2953f640 1 --2- 172.21.15.52:0/3667298707 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcf080678f0 0x7fcf08069db0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fcf24071780 tx=0x7fcf14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:34.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.143+0000 7fcf1a7fc640 1 -- 172.21.15.52:0/3667298707 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcf20016090 con 0x7fcf240cd060 2024-05-04T21:20:34.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.303+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/3667298707 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fcf24075320 con 0x7fcf080678f0 2024-05-04T21:20:34.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.303+0000 7fcf1a7fc640 1 -- 172.21.15.52:0/3667298707 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcf24075320 con 0x7fcf080678f0 2024-05-04T21:20:34.309 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 1 to deep-scrub 2024-05-04T21:20:34.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.307+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/3667298707 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcf080678f0 msgr2=0x7fcf08069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.307+0000 7fcf2bfa4640 1 --2- 172.21.15.52:0/3667298707 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcf080678f0 0x7fcf08069db0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fcf24071780 tx=0x7fcf14008040 comp rx=0 tx=0).stop 2024-05-04T21:20:34.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.307+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/3667298707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf240cd060 msgr2=0x7fcf241484a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.307+0000 7fcf2bfa4640 1 --2- 172.21.15.52:0/3667298707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf240cd060 0x7fcf241484a0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fcf200ae040 tx=0x7fcf2001ac20 comp rx=0 tx=0).stop 2024-05-04T21:20:34.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.307+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/3667298707 shutdown_connections 2024-05-04T21:20:34.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.307+0000 7fcf2bfa4640 1 --2- 172.21.15.52:0/3667298707 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcf080678f0 0x7fcf08069db0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.307+0000 7fcf2bfa4640 1 --2- 172.21.15.52:0/3667298707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf2415c520 0x7fcf24150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.307+0000 7fcf2bfa4640 1 --2- 172.21.15.52:0/3667298707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf240cf430 0x7fcf241469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.307+0000 7fcf2bfa4640 1 --2- 172.21.15.52:0/3667298707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf240cd060 0x7fcf241484a0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.307+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/3667298707 >> 172.21.15.52:0/3667298707 conn(0x7fcf240cd5b0 msgr2=0x7fcf2415ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:34.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.307+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/3667298707 shutdown_connections 2024-05-04T21:20:34.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.307+0000 7fcf2bfa4640 1 -- 172.21.15.52:0/3667298707 wait complete. 2024-05-04T21:20:34.328 INFO:tasks.ceph:Scrubbing osd.2 2024-05-04T21:20:34.329 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-04T21:20:34.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f7626b90640 1 Processor -- start 2024-05-04T21:20:34.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f7626b90640 1 -- start start 2024-05-04T21:20:34.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f7626b90640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7620158810 0x7f76200ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f7626b90640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7620162570 0x7f762006ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f7626b90640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7620071a50 con 0x7f76200addd0 2024-05-04T21:20:34.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f7626b90640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f7620162bd0 con 0x7f7620162570 2024-05-04T21:20:34.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f7626b90640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f76200ae230 con 0x7f7620158810 2024-05-04T21:20:34.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f762512d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7620162570 0x7f762006ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f761ffff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f76200addd0 0x7f7620161e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55476/0 (socket says 172.21.15.52:55476) 2024-05-04T21:20:34.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f761ffff640 1 -- 172.21.15.52:0/2594036304 learned_addr learned my addr 172.21.15.52:0/2594036304 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:34.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f762492c640 1 --2- 172.21.15.52:0/2594036304 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7620158810 0x7f76200ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f762512d640 1 -- 172.21.15.52:0/2594036304 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7620158810 msgr2=0x7f76200ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:34.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f762512d640 1 --2- 172.21.15.52:0/2594036304 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7620158810 0x7f76200ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f762512d640 1 -- 172.21.15.52:0/2594036304 >> v1:172.21.15.52:6789/0 conn(0x7f76200addd0 legacy=0x7f7620161e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f761f7fe640 1 -- 172.21.15.52:0/2594036304 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180692462 0 0) 0x7f7620071a50 con 0x7f76200addd0 2024-05-04T21:20:34.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.443+0000 7f762512d640 1 -- 172.21.15.52:0/2594036304 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7620071d30 con 0x7f7620162570 2024-05-04T21:20:34.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f762512d640 1 --2- 172.21.15.52:0/2594036304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7620162570 0x7f762006ce50 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f76100097c0 tx=0x7f761001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420707102ea684f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:34.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f761f7fe640 1 -- 172.21.15.52:0/2594036304 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76100b0070 con 0x7f7620162570 2024-05-04T21:20:34.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f761f7fe640 1 -- 172.21.15.52:0/2594036304 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76100ab600 con 0x7f7620162570 2024-05-04T21:20:34.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f761f7fe640 1 -- 172.21.15.52:0/2594036304 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76100ab9d0 con 0x7f7620162570 2024-05-04T21:20:34.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 -- 172.21.15.52:0/2594036304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7620162570 msgr2=0x7f762006ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 --2- 172.21.15.52:0/2594036304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7620162570 0x7f762006ce50 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f76100097c0 tx=0x7f761001c4a0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 -- 172.21.15.52:0/2594036304 shutdown_connections 2024-05-04T21:20:34.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 --2- 172.21.15.52:0/2594036304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7620162570 0x7f762006ce50 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 --2- 172.21.15.52:0/2594036304 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7620158810 0x7f76200ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 -- 172.21.15.52:0/2594036304 >> 172.21.15.52:0/2594036304 conn(0x7f7620067760 msgr2=0x7f76200b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:34.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 -- 172.21.15.52:0/2594036304 shutdown_connections 2024-05-04T21:20:34.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 -- 172.21.15.52:0/2594036304 wait complete. 2024-05-04T21:20:34.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 Processor -- start 2024-05-04T21:20:34.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 -- start start 2024-05-04T21:20:34.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76200addd0 0x7f7620152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7620158810 0x7f762015ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7620162570 0x7f762015bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f762006df30 con 0x7f7620162570 2024-05-04T21:20:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f762006ddb0 con 0x7f7620158810 2024-05-04T21:20:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f7626b90640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f7620072380 con 0x7f76200addd0 2024-05-04T21:20:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f762512d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7620162570 0x7f762015bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f762512d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7620162570 0x7f762015bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59838/0 (socket says 172.21.15.52:59838) 2024-05-04T21:20:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f762512d640 1 -- 172.21.15.52:0/4156354376 learned_addr learned my addr 172.21.15.52:0/4156354376 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f762492c640 1 --2- 172.21.15.52:0/4156354376 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76200addd0 0x7f7620152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f761ffff640 1 --2- 172.21.15.52:0/4156354376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7620158810 0x7f762015ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f762512d640 1 -- 172.21.15.52:0/4156354376 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76200addd0 msgr2=0x7f7620152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f762512d640 1 --2- 172.21.15.52:0/4156354376 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f76200addd0 0x7f7620152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f762512d640 1 -- 172.21.15.52:0/4156354376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7620158810 msgr2=0x7f762015ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f762512d640 1 --2- 172.21.15.52:0/4156354376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7620158810 0x7f762015ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f762512d640 1 -- 172.21.15.52:0/4156354376 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762015c570 con 0x7f7620162570 2024-05-04T21:20:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.447+0000 7f762512d640 1 --2- 172.21.15.52:0/4156354376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7620162570 0x7f762015bf40 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f76100ae040 tx=0x7f7610019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f761d7fa640 1 -- 172.21.15.52:0/4156354376 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76100b8020 con 0x7f7620162570 2024-05-04T21:20:34.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f761d7fa640 1 -- 172.21.15.52:0/4156354376 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76100b46d0 con 0x7f7620162570 2024-05-04T21:20:34.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f7626b90640 1 -- 172.21.15.52:0/4156354376 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7620158e40 con 0x7f7620162570 2024-05-04T21:20:34.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f761d7fa640 1 -- 172.21.15.52:0/4156354376 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76100b0040 con 0x7f7620162570 2024-05-04T21:20:34.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f7626b90640 1 -- 172.21.15.52:0/4156354376 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7620159400 con 0x7f7620162570 2024-05-04T21:20:34.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f7626b90640 1 -- 172.21.15.52:0/4156354376 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7620062e50 con 0x7f7620162570 2024-05-04T21:20:34.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f761d7fa640 1 -- 172.21.15.52:0/4156354376 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f76100bb020 con 0x7f7620162570 2024-05-04T21:20:34.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f761d7fa640 1 --2- 172.21.15.52:0/4156354376 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f76000678f0 0x7f7600069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f761d7fa640 1 -- 172.21.15.52:0/4156354376 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f761011f800 con 0x7f7620162570 2024-05-04T21:20:34.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f762492c640 1 --2- 172.21.15.52:0/4156354376 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f76000678f0 0x7f7600069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f761d7fa640 1 --2- 172.21.15.52:0/4156354376 >> [v2:172.21.15.52:6800/4058473938,v1:172.21.15.52:6801/4058473938] conn(0x7f760006d7b0 0x7f760006fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f761d7fa640 1 -- 172.21.15.52:0/4156354376 --> [v2:172.21.15.52:6800/4058473938,v1:172.21.15.52:6801/4058473938] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76100a9ec0 con 0x7f760006d7b0 2024-05-04T21:20:34.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f761d7fa640 1 -- 172.21.15.52:0/4156354376 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7610126050 con 0x7f7620162570 2024-05-04T21:20:34.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f761ffff640 1 --2- 172.21.15.52:0/4156354376 >> [v2:172.21.15.52:6800/4058473938,v1:172.21.15.52:6801/4058473938] conn(0x7f760006d7b0 0x7f760006fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f762492c640 1 --2- 172.21.15.52:0/4156354376 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f76000678f0 0x7f7600069db0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f76180070b0 tx=0x7f7618007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:34.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.451+0000 7f761ffff640 1 --2- 172.21.15.52:0/4156354376 >> [v2:172.21.15.52:6800/4058473938,v1:172.21.15.52:6801/4058473938] conn(0x7f760006d7b0 0x7f760006fbd0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:34.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.487+0000 7f761d7fa640 1 -- 172.21.15.52:0/4156354376 <== osd.2 v2:172.21.15.52:6800/4058473938 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76100a9ec0 con 0x7f760006d7b0 2024-05-04T21:20:34.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.499+0000 7f7626b90640 1 -- 172.21.15.52:0/4156354376 --> [v2:172.21.15.52:6800/4058473938,v1:172.21.15.52:6801/4058473938] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7620075320 con 0x7f760006d7b0 2024-05-04T21:20:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.511+0000 7f761d7fa640 1 -- 172.21.15.52:0/4156354376 <== osd.2 v2:172.21.15.52:6800/4058473938 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7620075320 con 0x7f760006d7b0 2024-05-04T21:20:34.516 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-04T21:20:34.516 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-04T21:20:34.516 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-04T21:20:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.511+0000 7f7626b90640 1 -- 172.21.15.52:0/4156354376 >> [v2:172.21.15.52:6800/4058473938,v1:172.21.15.52:6801/4058473938] conn(0x7f760006d7b0 msgr2=0x7f760006fbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.511+0000 7f7626b90640 1 --2- 172.21.15.52:0/4156354376 >> [v2:172.21.15.52:6800/4058473938,v1:172.21.15.52:6801/4058473938] conn(0x7f760006d7b0 0x7f760006fbd0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.511+0000 7f7626b90640 1 -- 172.21.15.52:0/4156354376 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f76000678f0 msgr2=0x7f7600069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.511+0000 7f7626b90640 1 --2- 172.21.15.52:0/4156354376 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f76000678f0 0x7f7600069db0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f76180070b0 tx=0x7f7618007040 comp rx=0 tx=0).stop 2024-05-04T21:20:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.511+0000 7f7626b90640 1 -- 172.21.15.52:0/4156354376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7620162570 msgr2=0x7f762015bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.511+0000 7f7626b90640 1 --2- 172.21.15.52:0/4156354376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7620162570 0x7f762015bf40 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f76100ae040 tx=0x7f7610019240 comp rx=0 tx=0).stop 2024-05-04T21:20:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.511+0000 7f762512d640 1 -- 172.21.15.52:0/4156354376 reap_dead start 2024-05-04T21:20:34.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.511+0000 7f7626b90640 1 -- 172.21.15.52:0/4156354376 shutdown_connections 2024-05-04T21:20:34.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.511+0000 7f7626b90640 1 -- 172.21.15.52:0/4156354376 >> 172.21.15.52:0/4156354376 conn(0x7f7620067760 msgr2=0x7f762015f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:34.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.511+0000 7f7626b90640 1 -- 172.21.15.52:0/4156354376 shutdown_connections 2024-05-04T21:20:34.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.511+0000 7f7626b90640 1 -- 172.21.15.52:0/4156354376 wait complete. 2024-05-04T21:20:34.537 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-04T21:20:34.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f042b0e6640 1 Processor -- start 2024-05-04T21:20:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f042b0e6640 1 -- start start 2024-05-04T21:20:34.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f042b0e6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f04240cc690 0x7f04240b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f042b0e6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f04240b0ca0 0x7f0424070770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f042b0e6640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f042406cf00 con 0x7f042415e470 2024-05-04T21:20:34.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f042b0e6640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f04240b1110 con 0x7f04240b0ca0 2024-05-04T21:20:34.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f042b0e6640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f042406d1e0 con 0x7f04240cc690 2024-05-04T21:20:34.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f0429683640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f04240b0ca0 0x7f0424070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f0429683640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f04240b0ca0 0x7f0424070770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59850/0 (socket says 172.21.15.52:59850) 2024-05-04T21:20:34.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f0429683640 1 -- 172.21.15.52:0/923335969 learned_addr learned my addr 172.21.15.52:0/923335969 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:34.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f0429683640 1 -- 172.21.15.52:0/923335969 >> v2:172.21.15.111:3300/0 conn(0x7f04240cc690 msgr2=0x7f04240b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:34.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f0423fff640 1 --2- 172.21.15.52:0/923335969 >> v2:172.21.15.111:3300/0 conn(0x7f04240cc690 0x7f04240b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f0429683640 1 --2- 172.21.15.52:0/923335969 >> v2:172.21.15.111:3300/0 conn(0x7f04240cc690 0x7f04240b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f0429683640 1 -- 172.21.15.52:0/923335969 >> v1:172.21.15.111:6789/0 conn(0x7f042415e470 legacy=0x7f042406e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:34.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.659+0000 7f0429683640 1 -- 172.21.15.52:0/923335969 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042406d4c0 con 0x7f04240b0ca0 2024-05-04T21:20:34.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f0429683640 1 --2- 172.21.15.52:0/923335969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04240b0ca0 0x7f0424070770 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f041c009e50 tx=0x7f041c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c5c21bdd8189cd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:34.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f04237fe640 1 -- 172.21.15.52:0/923335969 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c0a56f0 con 0x7f04240b0ca0 2024-05-04T21:20:34.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f04237fe640 1 -- 172.21.15.52:0/923335969 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f041c0a5890 con 0x7f04240b0ca0 2024-05-04T21:20:34.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f04237fe640 1 -- 172.21.15.52:0/923335969 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c0a5c60 con 0x7f04240b0ca0 2024-05-04T21:20:34.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 -- 172.21.15.52:0/923335969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04240b0ca0 msgr2=0x7f0424070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 --2- 172.21.15.52:0/923335969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04240b0ca0 0x7f0424070770 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f041c009e50 tx=0x7f041c01c920 comp rx=0 tx=0).stop 2024-05-04T21:20:34.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 -- 172.21.15.52:0/923335969 shutdown_connections 2024-05-04T21:20:34.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 --2- 172.21.15.52:0/923335969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04240b0ca0 0x7f0424070770 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 --2- 172.21.15.52:0/923335969 >> v2:172.21.15.111:3300/0 conn(0x7f04240cc690 0x7f04240b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 -- 172.21.15.52:0/923335969 >> 172.21.15.52:0/923335969 conn(0x7f04240b14a0 msgr2=0x7f04240b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:34.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 -- 172.21.15.52:0/923335969 shutdown_connections 2024-05-04T21:20:34.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 -- 172.21.15.52:0/923335969 wait complete. 2024-05-04T21:20:34.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 Processor -- start 2024-05-04T21:20:34.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 -- start start 2024-05-04T21:20:34.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04240b0ca0 0x7f042413e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04240cc690 0x7f042415c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f042415e470 0x7f04241564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0424071850 con 0x7f04240b0ca0 2024-05-04T21:20:34.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f04240716d0 con 0x7f042415e470 2024-05-04T21:20:34.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f042b0e6640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f04240719d0 con 0x7f04240cc690 2024-05-04T21:20:34.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f0428e82640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04240b0ca0 0x7f042413e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f0428e82640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04240b0ca0 0x7f042413e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59858/0 (socket says 172.21.15.52:59858) 2024-05-04T21:20:34.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f0428e82640 1 -- 172.21.15.52:0/3949861460 learned_addr learned my addr 172.21.15.52:0/3949861460 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:34.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f0429683640 1 --2- 172.21.15.52:0/3949861460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f042415e470 0x7f04241564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f0423fff640 1 --2- 172.21.15.52:0/3949861460 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04240cc690 0x7f042415c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f0428e82640 1 -- 172.21.15.52:0/3949861460 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04240cc690 msgr2=0x7f042415c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f0428e82640 1 --2- 172.21.15.52:0/3949861460 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04240cc690 0x7f042415c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f0428e82640 1 -- 172.21.15.52:0/3949861460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f042415e470 msgr2=0x7f04241564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f0428e82640 1 --2- 172.21.15.52:0/3949861460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f042415e470 0x7f04241564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.663+0000 7f0428e82640 1 -- 172.21.15.52:0/3949861460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042413e8c0 con 0x7f04240b0ca0 2024-05-04T21:20:34.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.667+0000 7f0428e82640 1 --2- 172.21.15.52:0/3949861460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04240b0ca0 0x7f042413e380 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f041400b410 tx=0x7f041400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:34.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.667+0000 7f04217fa640 1 -- 172.21.15.52:0/3949861460 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414015030 con 0x7f04240b0ca0 2024-05-04T21:20:34.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.667+0000 7f04217fa640 1 -- 172.21.15.52:0/3949861460 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0414004020 con 0x7f04240b0ca0 2024-05-04T21:20:34.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.667+0000 7f042b0e6640 1 -- 172.21.15.52:0/3949861460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042414fba0 con 0x7f04240b0ca0 2024-05-04T21:20:34.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.667+0000 7f04217fa640 1 -- 172.21.15.52:0/3949861460 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414004410 con 0x7f04240b0ca0 2024-05-04T21:20:34.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.667+0000 7f042b0e6640 1 -- 172.21.15.52:0/3949861460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0424150160 con 0x7f04240b0ca0 2024-05-04T21:20:34.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.667+0000 7f042b0e6640 1 -- 172.21.15.52:0/3949861460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0424062e50 con 0x7f04240b0ca0 2024-05-04T21:20:34.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.667+0000 7f04217fa640 1 -- 172.21.15.52:0/3949861460 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0414018070 con 0x7f04240b0ca0 2024-05-04T21:20:34.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.667+0000 7f04217fa640 1 --2- 172.21.15.52:0/3949861460 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f04100678c0 0x7f0410069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.667+0000 7f04217fa640 1 -- 172.21.15.52:0/3949861460 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f0414085610 con 0x7f04240b0ca0 2024-05-04T21:20:34.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.667+0000 7f0423fff640 1 --2- 172.21.15.52:0/3949861460 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f04100678c0 0x7f0410069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.667+0000 7f0423fff640 1 --2- 172.21.15.52:0/3949861460 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f04100678c0 0x7f0410069d80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0424157120 tx=0x7f0418008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:34.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.671+0000 7f04217fa640 1 -- 172.21.15.52:0/3949861460 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0414056320 con 0x7f04240b0ca0 2024-05-04T21:20:34.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.835+0000 7f042b0e6640 1 -- 172.21.15.52:0/3949861460 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f0424075320 con 0x7f04100678c0 2024-05-04T21:20:34.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.835+0000 7f04217fa640 1 -- 172.21.15.52:0/3949861460 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0424075320 con 0x7f04100678c0 2024-05-04T21:20:34.841 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 2 to deep-scrub 2024-05-04T21:20:34.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.839+0000 7f042b0e6640 1 -- 172.21.15.52:0/3949861460 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f04100678c0 msgr2=0x7f0410069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.839+0000 7f042b0e6640 1 --2- 172.21.15.52:0/3949861460 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f04100678c0 0x7f0410069d80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0424157120 tx=0x7f0418008040 comp rx=0 tx=0).stop 2024-05-04T21:20:34.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.839+0000 7f042b0e6640 1 -- 172.21.15.52:0/3949861460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04240b0ca0 msgr2=0x7f042413e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.839+0000 7f042b0e6640 1 --2- 172.21.15.52:0/3949861460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04240b0ca0 0x7f042413e380 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f041400b410 tx=0x7f041400b8d0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.839+0000 7f042b0e6640 1 -- 172.21.15.52:0/3949861460 shutdown_connections 2024-05-04T21:20:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.839+0000 7f042b0e6640 1 --2- 172.21.15.52:0/3949861460 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f04100678c0 0x7f0410069d80 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.839+0000 7f042b0e6640 1 --2- 172.21.15.52:0/3949861460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f042415e470 0x7f04241564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.839+0000 7f042b0e6640 1 --2- 172.21.15.52:0/3949861460 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04240cc690 0x7f042415c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.839+0000 7f042b0e6640 1 --2- 172.21.15.52:0/3949861460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04240b0ca0 0x7f042413e380 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.839+0000 7f042b0e6640 1 -- 172.21.15.52:0/3949861460 >> 172.21.15.52:0/3949861460 conn(0x7f04240b14a0 msgr2=0x7f04240cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.839+0000 7f042b0e6640 1 -- 172.21.15.52:0/3949861460 shutdown_connections 2024-05-04T21:20:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.839+0000 7f042b0e6640 1 -- 172.21.15.52:0/3949861460 wait complete. 2024-05-04T21:20:34.860 INFO:tasks.ceph:Scrubbing osd.3 2024-05-04T21:20:34.860 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-04T21:20:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dc55c2640 1 Processor -- start 2024-05-04T21:20:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dc55c2640 1 -- start start 2024-05-04T21:20:34.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dc55c2640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dc01553d0 0x7f5dc00cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dc55c2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5dc00cfa70 0x7f5dc015c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dc55c2640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc00cdeb0 con 0x7f5dc015c150 2024-05-04T21:20:34.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dc55c2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f5dc00cd2a0 con 0x7f5dc00cfa70 2024-05-04T21:20:34.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dc55c2640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc00cd580 con 0x7f5dc01553d0 2024-05-04T21:20:34.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dbe7fc640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f5dc015c150 0x7f5dc00cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55482/0 (socket says 172.21.15.52:55482) 2024-05-04T21:20:34.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dbe7fc640 1 -- 172.21.15.52:0/2728600850 learned_addr learned my addr 172.21.15.52:0/2728600850 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:34.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dbeffd640 1 --2- 172.21.15.52:0/2728600850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dc01553d0 0x7f5dc00cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dbf7fe640 1 --2- 172.21.15.52:0/2728600850 >> v2:172.21.15.111:3300/0 conn(0x7f5dc00cfa70 0x7f5dc015c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dbdffb640 1 -- 172.21.15.52:0/2728600850 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2703664218 0 0) 0x7f5dc00cdeb0 con 0x7f5dc015c150 2024-05-04T21:20:34.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dbdffb640 1 -- 172.21.15.52:0/2728600850 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5da8003530 con 0x7f5dc015c150 2024-05-04T21:20:34.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dbdffb640 1 -- 172.21.15.52:0/2728600850 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4034155626 0 0) 0x7f5da8003530 con 0x7f5dc015c150 2024-05-04T21:20:34.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dbdffb640 1 -- 172.21.15.52:0/2728600850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dc01553d0 msgr2=0x7f5dc00cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:34.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dbdffb640 1 --2- 172.21.15.52:0/2728600850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dc01553d0 0x7f5dc00cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dbdffb640 1 -- 172.21.15.52:0/2728600850 >> v2:172.21.15.111:3300/0 conn(0x7f5dc00cfa70 msgr2=0x7f5dc015c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:34.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dbdffb640 1 --2- 172.21.15.52:0/2728600850 >> v2:172.21.15.111:3300/0 conn(0x7f5dc00cfa70 0x7f5dc015c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dbdffb640 1 -- 172.21.15.52:0/2728600850 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc00cd860 con 0x7f5dc015c150 2024-05-04T21:20:34.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dbdffb640 1 -- 172.21.15.52:0/2728600850 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f5db4001a00 con 0x7f5dc015c150 2024-05-04T21:20:34.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.983+0000 7f5dbdffb640 1 -- 172.21.15.52:0/2728600850 >> v1:172.21.15.52:6789/0 conn(0x7f5dc015c150 legacy=0x7f5dc00cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:34.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbdffb640 1 --2- 172.21.15.52:0/2728600850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5da80039c0 0x7f5da8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbdffb640 1 --2- 172.21.15.52:0/2728600850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5da8004810 0x7f5da8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbdffb640 1 --2- 172.21.15.52:0/2728600850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dc01553d0 0x7f5da80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:34.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbdffb640 1 -- 172.21.15.52:0/2728600850 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc00cd2a0 con 0x7f5da80039c0 2024-05-04T21:20:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbdffb640 1 -- 172.21.15.52:0/2728600850 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc00cd580 con 0x7f5da8004810 2024-05-04T21:20:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbdffb640 1 -- 172.21.15.52:0/2728600850 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5da800b9d0 con 0x7f5dc01553d0 2024-05-04T21:20:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbeffd640 1 --2- 172.21.15.52:0/2728600850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5da80039c0 0x7f5da8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbeffd640 1 -- 172.21.15.52:0/2728600850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dc01553d0 msgr2=0x7f5da80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbeffd640 1 --2- 172.21.15.52:0/2728600850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dc01553d0 0x7f5da80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbeffd640 1 -- 172.21.15.52:0/2728600850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5da8004810 msgr2=0x7f5da8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbeffd640 1 --2- 172.21.15.52:0/2728600850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5da8004810 0x7f5da8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:34.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbeffd640 1 -- 172.21.15.52:0/2728600850 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da8003530 con 0x7f5da80039c0 2024-05-04T21:20:34.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbeffd640 1 --2- 172.21.15.52:0/2728600850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5da80039c0 0x7f5da8003de0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009670 tx=0x7f5db801ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e74984877a285f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:34.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbdffb640 1 -- 172.21.15.52:0/2728600850 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db80b4070 con 0x7f5da80039c0 2024-05-04T21:20:34.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbdffb640 1 -- 172.21.15.52:0/2728600850 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db80a76f0 con 0x7f5da80039c0 2024-05-04T21:20:35.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbdffb640 1 -- 172.21.15.52:0/2728600850 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db80096b0 con 0x7f5da80039c0 2024-05-04T21:20:35.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dc55c2640 1 -- 172.21.15.52:0/2728600850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5da80039c0 msgr2=0x7f5da8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dc55c2640 1 --2- 172.21.15.52:0/2728600850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5da80039c0 0x7f5da8003de0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009670 tx=0x7f5db801ebd0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dbf7fe640 1 -- 172.21.15.52:0/2728600850 reap_dead start 2024-05-04T21:20:35.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dc55c2640 1 -- 172.21.15.52:0/2728600850 shutdown_connections 2024-05-04T21:20:35.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dc55c2640 1 -- 172.21.15.52:0/2728600850 >> 172.21.15.52:0/2728600850 conn(0x7f5dc0067760 msgr2=0x7f5dc00b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:35.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dc55c2640 1 -- 172.21.15.52:0/2728600850 shutdown_connections 2024-05-04T21:20:35.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.987+0000 7f5dc55c2640 1 -- 172.21.15.52:0/2728600850 wait complete. 2024-05-04T21:20:35.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dc55c2640 1 Processor -- start 2024-05-04T21:20:35.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dc55c2640 1 -- start start 2024-05-04T21:20:35.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dc55c2640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5da80039c0 0x7f5dc006f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dc55c2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dc013b3d0 0x7f5dc006cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dc55c2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc0137fd0 0x7f5dc015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dc55c2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc015d190 con 0x7f5dc0137fd0 2024-05-04T21:20:35.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dc55c2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc015d010 con 0x7f5dc013b3d0 2024-05-04T21:20:35.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dc55c2640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc015d310 con 0x7f5da80039c0 2024-05-04T21:20:35.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dbf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc0137fd0 0x7f5dc015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dbf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc0137fd0 0x7f5dc015e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59882/0 (socket says 172.21.15.52:59882) 2024-05-04T21:20:35.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dbf7fe640 1 -- 172.21.15.52:0/1854326745 learned_addr learned my addr 172.21.15.52:0/1854326745 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:35.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dbe7fc640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dc013b3d0 0x7f5dc006cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dbeffd640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5da80039c0 0x7f5dc006f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dbf7fe640 1 -- 172.21.15.52:0/1854326745 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5da80039c0 msgr2=0x7f5dc006f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dbf7fe640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5da80039c0 0x7f5dc006f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dbf7fe640 1 -- 172.21.15.52:0/1854326745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dc013b3d0 msgr2=0x7f5dc006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dbf7fe640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dc013b3d0 0x7f5dc006cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dbf7fe640 1 -- 172.21.15.52:0/1854326745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc006e7d0 con 0x7f5dc0137fd0 2024-05-04T21:20:35.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dbe7fc640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dc013b3d0 0x7f5dc006cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:35.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dbeffd640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5da80039c0 0x7f5dc006f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:35.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dbf7fe640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc0137fd0 0x7f5dc015e9f0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00720a0 tx=0x7f5db000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:35.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5da77fe640 1 -- 172.21.15.52:0/1854326745 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db000c6b0 con 0x7f5dc0137fd0 2024-05-04T21:20:35.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5da77fe640 1 -- 172.21.15.52:0/1854326745 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db001a070 con 0x7f5dc0137fd0 2024-05-04T21:20:35.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dc55c2640 1 -- 172.21.15.52:0/1854326745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc006d350 con 0x7f5dc0137fd0 2024-05-04T21:20:35.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5da77fe640 1 -- 172.21.15.52:0/1854326745 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db000cab0 con 0x7f5dc0137fd0 2024-05-04T21:20:35.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dc55c2640 1 -- 172.21.15.52:0/1854326745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc006ea50 con 0x7f5dc0137fd0 2024-05-04T21:20:35.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5dc55c2640 1 -- 172.21.15.52:0/1854326745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5dc0062e50 con 0x7f5dc0137fd0 2024-05-04T21:20:35.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5da77fe640 1 -- 172.21.15.52:0/1854326745 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5db000cc50 con 0x7f5dc0137fd0 2024-05-04T21:20:35.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.991+0000 7f5da77fe640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5dac0678f0 0x7f5dac069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.995+0000 7f5da77fe640 1 -- 172.21.15.52:0/1854326745 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f5db00857c0 con 0x7f5dc0137fd0 2024-05-04T21:20:35.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.995+0000 7f5dbeffd640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5dac0678f0 0x7f5dac069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.995+0000 7f5da77fe640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.52:6824/2130169459,v1:172.21.15.52:6825/2130169459] conn(0x7f5dac06d7b0 0x7f5dac06fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.995+0000 7f5da77fe640 1 -- 172.21.15.52:0/1854326745 --> [v2:172.21.15.52:6824/2130169459,v1:172.21.15.52:6825/2130169459] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5db0015e50 con 0x7f5dac06d7b0 2024-05-04T21:20:35.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.995+0000 7f5da77fe640 1 -- 172.21.15.52:0/1854326745 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5db0085c50 con 0x7f5dc0137fd0 2024-05-04T21:20:35.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.995+0000 7f5dbe7fc640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.52:6824/2130169459,v1:172.21.15.52:6825/2130169459] conn(0x7f5dac06d7b0 0x7f5dac06fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.995+0000 7f5dbeffd640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5dac0678f0 0x7f5dac069db0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5db80b1000 tx=0x7f5db80046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:35.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:34.995+0000 7f5dbe7fc640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.52:6824/2130169459,v1:172.21.15.52:6825/2130169459] conn(0x7f5dac06d7b0 0x7f5dac06fbd0 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:35.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.051+0000 7f5da77fe640 1 -- 172.21.15.52:0/1854326745 <== osd.3 v2:172.21.15.52:6824/2130169459 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5db0015e50 con 0x7f5dac06d7b0 2024-05-04T21:20:35.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.067+0000 7f5dc55c2640 1 -- 172.21.15.52:0/1854326745 --> [v2:172.21.15.52:6824/2130169459,v1:172.21.15.52:6825/2130169459] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5dc0075320 con 0x7f5dac06d7b0 2024-05-04T21:20:35.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.079+0000 7f5da77fe640 1 -- 172.21.15.52:0/1854326745 <== osd.3 v2:172.21.15.52:6824/2130169459 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5dc0075320 con 0x7f5dac06d7b0 2024-05-04T21:20:35.083 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-04T21:20:35.084 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-04T21:20:35.084 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-04T21:20:35.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.079+0000 7f5dc55c2640 1 -- 172.21.15.52:0/1854326745 >> [v2:172.21.15.52:6824/2130169459,v1:172.21.15.52:6825/2130169459] conn(0x7f5dac06d7b0 msgr2=0x7f5dac06fbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.079+0000 7f5dc55c2640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.52:6824/2130169459,v1:172.21.15.52:6825/2130169459] conn(0x7f5dac06d7b0 0x7f5dac06fbd0 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.079+0000 7f5dc55c2640 1 -- 172.21.15.52:0/1854326745 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5dac0678f0 msgr2=0x7f5dac069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.079+0000 7f5dc55c2640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f5dac0678f0 0x7f5dac069db0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5db80b1000 tx=0x7f5db80046d0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.079+0000 7f5dc55c2640 1 -- 172.21.15.52:0/1854326745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc0137fd0 msgr2=0x7f5dc015e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.079+0000 7f5dc55c2640 1 --2- 172.21.15.52:0/1854326745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc0137fd0 0x7f5dc015e9f0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00720a0 tx=0x7f5db000ec00 comp rx=0 tx=0).stop 2024-05-04T21:20:35.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.079+0000 7f5dbf7fe640 1 -- 172.21.15.52:0/1854326745 reap_dead start 2024-05-04T21:20:35.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.079+0000 7f5dc55c2640 1 -- 172.21.15.52:0/1854326745 shutdown_connections 2024-05-04T21:20:35.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.079+0000 7f5dc55c2640 1 -- 172.21.15.52:0/1854326745 >> 172.21.15.52:0/1854326745 conn(0x7f5dc0067760 msgr2=0x7f5dc00ae750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:35.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.079+0000 7f5dc55c2640 1 -- 172.21.15.52:0/1854326745 shutdown_connections 2024-05-04T21:20:35.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.079+0000 7f5dc55c2640 1 -- 172.21.15.52:0/1854326745 wait complete. 2024-05-04T21:20:35.108 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-04T21:20:35.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec3f46640 1 Processor -- start 2024-05-04T21:20:35.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec3f46640 1 -- start start 2024-05-04T21:20:35.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec3f46640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdebc0aa0c0 0x7fdebc0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec3f46640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdebc0aa550 0x7fdebc0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec3f46640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdebc06d1b0 con 0x7fdebc15bf50 2024-05-04T21:20:35.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec3f46640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fdebc15ca40 con 0x7fdebc0aa0c0 2024-05-04T21:20:35.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec3f46640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fdebc06d490 con 0x7fdebc0aa550 2024-05-04T21:20:35.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec14e1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdebc0aa0c0 0x7fdebc0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec14e1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdebc0aa0c0 0x7fdebc0ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59886/0 (socket says 172.21.15.52:59886) 2024-05-04T21:20:35.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec14e1640 1 -- 172.21.15.52:0/1098956448 learned_addr learned my addr 172.21.15.52:0/1098956448 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:35.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec24e3640 1 --2- 172.21.15.52:0/1098956448 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdebc0aa550 0x7fdebc0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec14e1640 1 -- 172.21.15.52:0/1098956448 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdebc0aa550 msgr2=0x7fdebc0cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:35.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec14e1640 1 --2- 172.21.15.52:0/1098956448 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdebc0aa550 0x7fdebc0cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec14e1640 1 -- 172.21.15.52:0/1098956448 >> v1:172.21.15.111:6789/0 conn(0x7fdebc15bf50 legacy=0x7fdebc15c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.235+0000 7fdec14e1640 1 -- 172.21.15.52:0/1098956448 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc06d770 con 0x7fdebc0aa0c0 2024-05-04T21:20:35.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.239+0000 7fdec14e1640 1 --2- 172.21.15.52:0/1098956448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdebc0aa0c0 0x7fdebc0ccb50 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009870 tx=0x7fdeac01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2742403a4038b76 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:35.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.239+0000 7fdec0ce0640 1 -- 172.21.15.52:0/1098956448 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0a9610 con 0x7fdebc0aa0c0 2024-05-04T21:20:35.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.239+0000 7fdec0ce0640 1 -- 172.21.15.52:0/1098956448 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeac0a97b0 con 0x7fdebc0aa0c0 2024-05-04T21:20:35.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.239+0000 7fdec0ce0640 1 -- 172.21.15.52:0/1098956448 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0a9b80 con 0x7fdebc0aa0c0 2024-05-04T21:20:35.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.239+0000 7fdec3f46640 1 -- 172.21.15.52:0/1098956448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdebc0aa0c0 msgr2=0x7fdebc0ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.239+0000 7fdec3f46640 1 --2- 172.21.15.52:0/1098956448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdebc0aa0c0 0x7fdebc0ccb50 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009870 tx=0x7fdeac01c530 comp rx=0 tx=0).stop 2024-05-04T21:20:35.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.239+0000 7fdec3f46640 1 -- 172.21.15.52:0/1098956448 shutdown_connections 2024-05-04T21:20:35.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.239+0000 7fdec3f46640 1 --2- 172.21.15.52:0/1098956448 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdebc0aa550 0x7fdebc0cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.239+0000 7fdec3f46640 1 --2- 172.21.15.52:0/1098956448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdebc0aa0c0 0x7fdebc0ccb50 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.239+0000 7fdec3f46640 1 -- 172.21.15.52:0/1098956448 >> 172.21.15.52:0/1098956448 conn(0x7fdebc0b14a0 msgr2=0x7fdebc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:35.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.239+0000 7fdec3f46640 1 -- 172.21.15.52:0/1098956448 shutdown_connections 2024-05-04T21:20:35.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.239+0000 7fdec3f46640 1 -- 172.21.15.52:0/1098956448 wait complete. 2024-05-04T21:20:35.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec3f46640 1 Processor -- start 2024-05-04T21:20:35.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec3f46640 1 -- start start 2024-05-04T21:20:35.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec3f46640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc0aa0c0 0x7fdebc151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec3f46640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdebc0aa550 0x7fdebc153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec3f46640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdebc15bf50 0x7fdebc1496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec3f46640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc06ebe0 con 0x7fdebc0aa550 2024-05-04T21:20:35.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec3f46640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc06ea60 con 0x7fdebc0aa0c0 2024-05-04T21:20:35.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec3f46640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fdebc06ed60 con 0x7fdebc15bf50 2024-05-04T21:20:35.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec14e1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdebc0aa550 0x7fdebc153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec14e1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdebc0aa550 0x7fdebc153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59900/0 (socket says 172.21.15.52:59900) 2024-05-04T21:20:35.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec14e1640 1 -- 172.21.15.52:0/3656162456 learned_addr learned my addr 172.21.15.52:0/3656162456 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:35.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec1ce2640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc0aa0c0 0x7fdebc151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec24e3640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdebc15bf50 0x7fdebc1496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec14e1640 1 -- 172.21.15.52:0/3656162456 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdebc15bf50 msgr2=0x7fdebc1496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec14e1640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdebc15bf50 0x7fdebc1496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec14e1640 1 -- 172.21.15.52:0/3656162456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc0aa0c0 msgr2=0x7fdebc151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec14e1640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc0aa0c0 0x7fdebc151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec14e1640 1 -- 172.21.15.52:0/3656162456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc1500c0 con 0x7fdebc0aa550 2024-05-04T21:20:35.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec24e3640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdebc15bf50 0x7fdebc1496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:35.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec14e1640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdebc0aa550 0x7fdebc153410 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0ab040 tx=0x7fdeac01ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:35.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdeb27fc640 1 -- 172.21.15.52:0/3656162456 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0a9610 con 0x7fdebc0aa550 2024-05-04T21:20:35.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdeb27fc640 1 -- 172.21.15.52:0/3656162456 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeac0bc670 con 0x7fdebc0aa550 2024-05-04T21:20:35.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdeb27fc640 1 -- 172.21.15.52:0/3656162456 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0bca60 con 0x7fdebc0aa550 2024-05-04T21:20:35.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec3f46640 1 -- 172.21.15.52:0/3656162456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdebc1503a0 con 0x7fdebc0aa550 2024-05-04T21:20:35.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec3f46640 1 -- 172.21.15.52:0/3656162456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc150960 con 0x7fdebc0aa550 2024-05-04T21:20:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec3f46640 1 -- 172.21.15.52:0/3656162456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdebc062e50 con 0x7fdebc0aa550 2024-05-04T21:20:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdeb27fc640 1 -- 172.21.15.52:0/3656162456 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdeac0c53f0 con 0x7fdebc0aa550 2024-05-04T21:20:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdeb27fc640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdea00678c0 0x7fdea0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdeb27fc640 1 -- 172.21.15.52:0/3656162456 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fdeac120060 con 0x7fdebc0aa550 2024-05-04T21:20:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.243+0000 7fdec1ce2640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdea00678c0 0x7fdea0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.247+0000 7fdec1ce2640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdea00678c0 0x7fdea0069d80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80015b0 tx=0x7fdeb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:35.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.251+0000 7fdeb27fc640 1 -- 172.21.15.52:0/3656162456 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdeac0ad020 con 0x7fdebc0aa550 2024-05-04T21:20:35.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.411+0000 7fdec3f46640 1 -- 172.21.15.52:0/3656162456 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fdebc075320 con 0x7fdea00678c0 2024-05-04T21:20:35.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.411+0000 7fdeb27fc640 1 -- 172.21.15.52:0/3656162456 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdebc075320 con 0x7fdea00678c0 2024-05-04T21:20:35.417 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 3 to deep-scrub 2024-05-04T21:20:35.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.415+0000 7fdec3f46640 1 -- 172.21.15.52:0/3656162456 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdea00678c0 msgr2=0x7fdea0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.415+0000 7fdec3f46640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdea00678c0 0x7fdea0069d80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80015b0 tx=0x7fdeb8009040 comp rx=0 tx=0).stop 2024-05-04T21:20:35.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.415+0000 7fdec3f46640 1 -- 172.21.15.52:0/3656162456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdebc0aa550 msgr2=0x7fdebc153410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.415+0000 7fdec3f46640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdebc0aa550 0x7fdebc153410 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0ab040 tx=0x7fdeac01ad70 comp rx=0 tx=0).stop 2024-05-04T21:20:35.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.415+0000 7fdec3f46640 1 -- 172.21.15.52:0/3656162456 shutdown_connections 2024-05-04T21:20:35.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.415+0000 7fdec3f46640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fdea00678c0 0x7fdea0069d80 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.415+0000 7fdec3f46640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdebc15bf50 0x7fdebc1496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.415+0000 7fdec3f46640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdebc0aa550 0x7fdebc153410 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.415+0000 7fdec3f46640 1 --2- 172.21.15.52:0/3656162456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdebc0aa0c0 0x7fdebc151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.415+0000 7fdec3f46640 1 -- 172.21.15.52:0/3656162456 >> 172.21.15.52:0/3656162456 conn(0x7fdebc0b14a0 msgr2=0x7fdebc0cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:35.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.415+0000 7fdec3f46640 1 -- 172.21.15.52:0/3656162456 shutdown_connections 2024-05-04T21:20:35.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.415+0000 7fdec3f46640 1 -- 172.21.15.52:0/3656162456 wait complete. 2024-05-04T21:20:35.437 INFO:tasks.ceph:Scrubbing osd.4 2024-05-04T21:20:35.438 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-04T21:20:35.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b54f4f640 1 Processor -- start 2024-05-04T21:20:35.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b54f4f640 1 -- start start 2024-05-04T21:20:35.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b54f4f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4b500c3d80 0x7f4b500c4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b54f4f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5006ecd0 0x7f4b5006f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b54f4f640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5006df10 con 0x7f4b500b2d50 2024-05-04T21:20:35.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b54f4f640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5006e1f0 con 0x7f4b500c3d80 2024-05-04T21:20:35.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b54f4f640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5006e4d0 con 0x7f4b5006ecd0 2024-05-04T21:20:35.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b4dd9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4b500c3d80 0x7f4b500c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b4dd9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4b500c3d80 0x7f4b500c4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59906/0 (socket says 172.21.15.52:59906) 2024-05-04T21:20:35.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b4dd9b640 1 -- 172.21.15.52:0/3970407113 learned_addr learned my addr 172.21.15.52:0/3970407113 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:35.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b4ed9d640 1 --2- 172.21.15.52:0/3970407113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5006ecd0 0x7f4b5006f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b4dd9b640 1 -- 172.21.15.52:0/3970407113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5006ecd0 msgr2=0x7f4b5006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:35.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b4dd9b640 1 --2- 172.21.15.52:0/3970407113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5006ecd0 0x7f4b5006f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b4dd9b640 1 -- 172.21.15.52:0/3970407113 >> v1:172.21.15.52:6789/0 conn(0x7f4b500b2d50 legacy=0x7f4b5015c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b4dd9b640 1 -- 172.21.15.52:0/3970407113 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5006e7b0 con 0x7f4b500c3d80 2024-05-04T21:20:35.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.563+0000 7f4b4d59a640 1 -- 172.21.15.52:0/3970407113 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2591152678 0 0) 0x7f4b5006df10 con 0x7f4b500b2d50 2024-05-04T21:20:35.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.567+0000 7f4b4dd9b640 1 --2- 172.21.15.52:0/3970407113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b500c3d80 0x7f4b500c4210 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009870 tx=0x7f4b4401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d1f955594a9036 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:35.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.567+0000 7f4b4d59a640 1 -- 172.21.15.52:0/3970407113 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440b2070 con 0x7f4b500c3d80 2024-05-04T21:20:35.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.567+0000 7f4b4d59a640 1 -- 172.21.15.52:0/3970407113 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b440a9070 con 0x7f4b500c3d80 2024-05-04T21:20:35.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.567+0000 7f4b4d59a640 1 -- 172.21.15.52:0/3970407113 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440ad880 con 0x7f4b500c3d80 2024-05-04T21:20:35.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.567+0000 7f4b54f4f640 1 -- 172.21.15.52:0/3970407113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b500c3d80 msgr2=0x7f4b500c4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.567+0000 7f4b54f4f640 1 --2- 172.21.15.52:0/3970407113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b500c3d80 0x7f4b500c4210 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009870 tx=0x7f4b4401c560 comp rx=0 tx=0).stop 2024-05-04T21:20:35.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.567+0000 7f4b54f4f640 1 -- 172.21.15.52:0/3970407113 shutdown_connections 2024-05-04T21:20:35.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.567+0000 7f4b54f4f640 1 --2- 172.21.15.52:0/3970407113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5006ecd0 0x7f4b5006f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.567+0000 7f4b54f4f640 1 --2- 172.21.15.52:0/3970407113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b500c3d80 0x7f4b500c4210 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.567+0000 7f4b54f4f640 1 -- 172.21.15.52:0/3970407113 >> 172.21.15.52:0/3970407113 conn(0x7f4b5006c450 msgr2=0x7f4b50067070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:35.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.567+0000 7f4b54f4f640 1 -- 172.21.15.52:0/3970407113 shutdown_connections 2024-05-04T21:20:35.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.567+0000 7f4b54f4f640 1 -- 172.21.15.52:0/3970407113 wait complete. 2024-05-04T21:20:35.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b54f4f640 1 Processor -- start 2024-05-04T21:20:35.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b54f4f640 1 -- start start 2024-05-04T21:20:35.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b54f4f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b5006ecd0 0x7f4b500d0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b54f4f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b500b2d50 0x7f4b500706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b54f4f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b500c3d80 0x7f4b5015dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b54f4f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50141370 con 0x7f4b5006ecd0 2024-05-04T21:20:35.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b54f4f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501411f0 con 0x7f4b500c3d80 2024-05-04T21:20:35.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b54f4f640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4b50167bb0 con 0x7f4b500b2d50 2024-05-04T21:20:35.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b4e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b5006ecd0 0x7f4b500d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b4e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b5006ecd0 0x7f4b500d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59916/0 (socket says 172.21.15.52:59916) 2024-05-04T21:20:35.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b4e59c640 1 -- 172.21.15.52:0/2998605892 learned_addr learned my addr 172.21.15.52:0/2998605892 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:35.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b4dd9b640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b500b2d50 0x7f4b500706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b4ed9d640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b500c3d80 0x7f4b5015dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b4e59c640 1 -- 172.21.15.52:0/2998605892 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b500b2d50 msgr2=0x7f4b500706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b4e59c640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b500b2d50 0x7f4b500706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b4e59c640 1 -- 172.21.15.52:0/2998605892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b500c3d80 msgr2=0x7f4b5015dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b4e59c640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b500c3d80 0x7f4b5015dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b4e59c640 1 -- 172.21.15.52:0/2998605892 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b50159130 con 0x7f4b5006ecd0 2024-05-04T21:20:35.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b4dd9b640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b500b2d50 0x7f4b500706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:35.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b4ed9d640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b500c3d80 0x7f4b5015dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:35.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b4e59c640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b5006ecd0 0x7f4b500d0810 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000c450 tx=0x7f4b4000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:35.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b3effd640 1 -- 172.21.15.52:0/2998605892 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40017070 con 0x7f4b5006ecd0 2024-05-04T21:20:35.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b3effd640 1 -- 172.21.15.52:0/2998605892 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b40005480 con 0x7f4b5006ecd0 2024-05-04T21:20:35.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b54f4f640 1 -- 172.21.15.52:0/2998605892 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b50159410 con 0x7f4b5006ecd0 2024-05-04T21:20:35.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b3effd640 1 -- 172.21.15.52:0/2998605892 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40003840 con 0x7f4b5006ecd0 2024-05-04T21:20:35.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b54f4f640 1 -- 172.21.15.52:0/2998605892 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b501599d0 con 0x7f4b5006ecd0 2024-05-04T21:20:35.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b54f4f640 1 -- 172.21.15.52:0/2998605892 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4b50062e50 con 0x7f4b5006ecd0 2024-05-04T21:20:35.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b3effd640 1 -- 172.21.15.52:0/2998605892 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4b4001a070 con 0x7f4b5006ecd0 2024-05-04T21:20:35.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b3effd640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4b300678c0 0x7f4b30069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.571+0000 7f4b3effd640 1 -- 172.21.15.52:0/2998605892 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f4b40085110 con 0x7f4b5006ecd0 2024-05-04T21:20:35.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.575+0000 7f4b4dd9b640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4b300678c0 0x7f4b30069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.575+0000 7f4b3effd640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.111:6823/2919151955,v1:172.21.15.111:6825/2919151955] conn(0x7f4b3006d780 0x7f4b3006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.575+0000 7f4b3effd640 1 -- 172.21.15.52:0/2998605892 --> [v2:172.21.15.111:6823/2919151955,v1:172.21.15.111:6825/2919151955] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b4000bf30 con 0x7f4b3006d780 2024-05-04T21:20:35.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.575+0000 7f4b3effd640 1 -- 172.21.15.52:0/2998605892 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4b400855a0 con 0x7f4b5006ecd0 2024-05-04T21:20:35.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.575+0000 7f4b4dd9b640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4b300678c0 0x7f4b30069d80 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4b440af000 tx=0x7f4b44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:35.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.575+0000 7f4b4ed9d640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.111:6823/2919151955,v1:172.21.15.111:6825/2919151955] conn(0x7f4b3006d780 0x7f4b3006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.575+0000 7f4b4ed9d640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.111:6823/2919151955,v1:172.21.15.111:6825/2919151955] conn(0x7f4b3006d780 0x7f4b3006fba0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:35.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.663+0000 7f4b3effd640 1 -- 172.21.15.52:0/2998605892 <== osd.4 v2:172.21.15.111:6823/2919151955 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b4000bf30 con 0x7f4b3006d780 2024-05-04T21:20:35.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.687+0000 7f4b54f4f640 1 -- 172.21.15.52:0/2998605892 --> [v2:172.21.15.111:6823/2919151955,v1:172.21.15.111:6825/2919151955] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4b50075320 con 0x7f4b3006d780 2024-05-04T21:20:35.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.723+0000 7f4b3effd640 1 -- 172.21.15.52:0/2998605892 <== osd.4 v2:172.21.15.111:6823/2919151955 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4b50075320 con 0x7f4b3006d780 2024-05-04T21:20:35.730 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-04T21:20:35.731 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-04T21:20:35.731 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-04T21:20:35.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.727+0000 7f4b54f4f640 1 -- 172.21.15.52:0/2998605892 >> [v2:172.21.15.111:6823/2919151955,v1:172.21.15.111:6825/2919151955] conn(0x7f4b3006d780 msgr2=0x7f4b3006fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.727+0000 7f4b54f4f640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.111:6823/2919151955,v1:172.21.15.111:6825/2919151955] conn(0x7f4b3006d780 0x7f4b3006fba0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.727+0000 7f4b54f4f640 1 -- 172.21.15.52:0/2998605892 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4b300678c0 msgr2=0x7f4b30069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.727+0000 7f4b54f4f640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4b300678c0 0x7f4b30069d80 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4b440af000 tx=0x7f4b44008040 comp rx=0 tx=0).stop 2024-05-04T21:20:35.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.727+0000 7f4b54f4f640 1 -- 172.21.15.52:0/2998605892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b5006ecd0 msgr2=0x7f4b500d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.727+0000 7f4b54f4f640 1 --2- 172.21.15.52:0/2998605892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b5006ecd0 0x7f4b500d0810 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000c450 tx=0x7f4b4000c910 comp rx=0 tx=0).stop 2024-05-04T21:20:35.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.727+0000 7f4b4ed9d640 1 -- 172.21.15.52:0/2998605892 reap_dead start 2024-05-04T21:20:35.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.727+0000 7f4b54f4f640 1 -- 172.21.15.52:0/2998605892 shutdown_connections 2024-05-04T21:20:35.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.727+0000 7f4b54f4f640 1 -- 172.21.15.52:0/2998605892 >> 172.21.15.52:0/2998605892 conn(0x7f4b5006c450 msgr2=0x7f4b5015f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:35.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.727+0000 7f4b54f4f640 1 -- 172.21.15.52:0/2998605892 shutdown_connections 2024-05-04T21:20:35.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.727+0000 7f4b54f4f640 1 -- 172.21.15.52:0/2998605892 wait complete. 2024-05-04T21:20:35.757 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-04T21:20:35.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.883+0000 7fc6b2aca640 1 Processor -- start 2024-05-04T21:20:35.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.883+0000 7fc6b2aca640 1 -- start start 2024-05-04T21:20:35.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.883+0000 7fc6b2aca640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc6ac15e5f0 0x7fc6ac0cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.883+0000 7fc6b2aca640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6ac0cd5f0 0x7fc6ac15f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.883+0000 7fc6b2aca640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ac15ef30 con 0x7fc6ac15f950 2024-05-04T21:20:35.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.883+0000 7fc6b2aca640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc6ac0b2200 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.883+0000 7fc6b2aca640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ac0b1960 con 0x7fc6ac0cd5f0 2024-05-04T21:20:35.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.887+0000 7fc6b0866640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc6ac15e5f0 0x7fc6ac0cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.887+0000 7fc6b0866640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc6ac15e5f0 0x7fc6ac0cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59926/0 (socket says 172.21.15.52:59926) 2024-05-04T21:20:35.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.887+0000 7fc6b0866640 1 -- 172.21.15.52:0/952232074 learned_addr learned my addr 172.21.15.52:0/952232074 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:35.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.887+0000 7fc6abfff640 1 --2- 172.21.15.52:0/952232074 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6ac0cd5f0 0x7fc6ac15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.887+0000 7fc6b0866640 1 -- 172.21.15.52:0/952232074 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6ac0cd5f0 msgr2=0x7fc6ac15f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:35.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.887+0000 7fc6b0866640 1 --2- 172.21.15.52:0/952232074 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6ac0cd5f0 0x7fc6ac15f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.887+0000 7fc6b0866640 1 -- 172.21.15.52:0/952232074 >> v1:172.21.15.111:6789/0 conn(0x7fc6ac15f950 legacy=0x7fc6ac06b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.887+0000 7fc6b0866640 1 -- 172.21.15.52:0/952232074 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac0b1c40 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.887+0000 7fc6b0866640 1 --2- 172.21.15.52:0/952232074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6ac15e5f0 0x7fc6ac0cd0b0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009870 tx=0x7fc69c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df20f9225f0e8a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:35.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.887+0000 7fc6ab7fe640 1 -- 172.21.15.52:0/952232074 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c0a7070 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.887+0000 7fc6ab7fe640 1 -- 172.21.15.52:0/952232074 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc69c0b0070 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.887+0000 7fc6ab7fe640 1 -- 172.21.15.52:0/952232074 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c0ab890 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 -- 172.21.15.52:0/952232074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6ac15e5f0 msgr2=0x7fc6ac0cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 --2- 172.21.15.52:0/952232074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6ac15e5f0 0x7fc6ac0cd0b0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009870 tx=0x7fc69c01c560 comp rx=0 tx=0).stop 2024-05-04T21:20:35.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 -- 172.21.15.52:0/952232074 shutdown_connections 2024-05-04T21:20:35.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 --2- 172.21.15.52:0/952232074 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6ac0cd5f0 0x7fc6ac15f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 --2- 172.21.15.52:0/952232074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6ac15e5f0 0x7fc6ac0cd0b0 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 -- 172.21.15.52:0/952232074 >> 172.21.15.52:0/952232074 conn(0x7fc6ac0b2380 msgr2=0x7fc6ac0b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:35.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 -- 172.21.15.52:0/952232074 shutdown_connections 2024-05-04T21:20:35.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 -- 172.21.15.52:0/952232074 wait complete. 2024-05-04T21:20:35.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 Processor -- start 2024-05-04T21:20:35.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 -- start start 2024-05-04T21:20:35.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6ac0cd5f0 0x7fc6ac143f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6ac15e5f0 0x7fc6ac15a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6ac15f950 0x7fc6ac15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac0b1690 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac0b1510 con 0x7fc6ac0cd5f0 2024-05-04T21:20:35.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ac06bbc0 con 0x7fc6ac15f950 2024-05-04T21:20:35.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6abfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6ac15e5f0 0x7fc6ac15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6abfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6ac15e5f0 0x7fc6ac15a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59940/0 (socket says 172.21.15.52:59940) 2024-05-04T21:20:35.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6abfff640 1 -- 172.21.15.52:0/2794280198 learned_addr learned my addr 172.21.15.52:0/2794280198 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:35.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b0866640 1 --2- 172.21.15.52:0/2794280198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6ac0cd5f0 0x7fc6ac143f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b1067640 1 --2- 172.21.15.52:0/2794280198 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6ac15f950 0x7fc6ac15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6abfff640 1 -- 172.21.15.52:0/2794280198 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6ac15f950 msgr2=0x7fc6ac15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6abfff640 1 --2- 172.21.15.52:0/2794280198 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6ac15f950 0x7fc6ac15a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6abfff640 1 -- 172.21.15.52:0/2794280198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6ac0cd5f0 msgr2=0x7fc6ac143f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:35.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6abfff640 1 --2- 172.21.15.52:0/2794280198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6ac0cd5f0 0x7fc6ac143f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:35.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6abfff640 1 -- 172.21.15.52:0/2794280198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac0ce9e0 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6abfff640 1 --2- 172.21.15.52:0/2794280198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6ac15e5f0 0x7fc6ac15a260 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000d830 tx=0x7fc6a000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:35.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6a97fa640 1 -- 172.21.15.52:0/2794280198 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a000b6c0 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6a97fa640 1 -- 172.21.15.52:0/2794280198 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a0019070 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6a97fa640 1 -- 172.21.15.52:0/2794280198 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a000ba60 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 -- 172.21.15.52:0/2794280198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ac1fbcc0 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.891+0000 7fc6b2aca640 1 -- 172.21.15.52:0/2794280198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ac144670 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.895+0000 7fc6b2aca640 1 -- 172.21.15.52:0/2794280198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ac062e50 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.895+0000 7fc6a97fa640 1 -- 172.21.15.52:0/2794280198 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc6a001e460 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.895+0000 7fc6a97fa640 1 --2- 172.21.15.52:0/2794280198 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc6980678c0 0x7fc698069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:35.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.895+0000 7fc6a97fa640 1 -- 172.21.15.52:0/2794280198 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fc6a0086090 con 0x7fc6ac15e5f0 2024-05-04T21:20:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.895+0000 7fc6b0866640 1 --2- 172.21.15.52:0/2794280198 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc6980678c0 0x7fc698069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.895+0000 7fc6b0866640 1 --2- 172.21.15.52:0/2794280198 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc6980678c0 0x7fc698069d80 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0af000 tx=0x7fc69c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:35.899+0000 7fc6a97fa640 1 -- 172.21.15.52:0/2794280198 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc6a0056ed0 con 0x7fc6ac15e5f0 2024-05-04T21:20:36.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.059+0000 7fc6b2aca640 1 -- 172.21.15.52:0/2794280198 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fc6ac0735e0 con 0x7fc6980678c0 2024-05-04T21:20:36.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.075+0000 7fc6a97fa640 1 -- 172.21.15.52:0/2794280198 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc6ac0735e0 con 0x7fc6980678c0 2024-05-04T21:20:36.081 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 4 to deep-scrub 2024-05-04T21:20:36.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.079+0000 7fc6b2aca640 1 -- 172.21.15.52:0/2794280198 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc6980678c0 msgr2=0x7fc698069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.079+0000 7fc6b2aca640 1 --2- 172.21.15.52:0/2794280198 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc6980678c0 0x7fc698069d80 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0af000 tx=0x7fc69c0a7000 comp rx=0 tx=0).stop 2024-05-04T21:20:36.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.079+0000 7fc6b2aca640 1 -- 172.21.15.52:0/2794280198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6ac15e5f0 msgr2=0x7fc6ac15a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.079+0000 7fc6b2aca640 1 --2- 172.21.15.52:0/2794280198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6ac15e5f0 0x7fc6ac15a260 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000d830 tx=0x7fc6a000dcf0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.079+0000 7fc6b2aca640 1 -- 172.21.15.52:0/2794280198 shutdown_connections 2024-05-04T21:20:36.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.079+0000 7fc6b2aca640 1 --2- 172.21.15.52:0/2794280198 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc6980678c0 0x7fc698069d80 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.079+0000 7fc6b2aca640 1 --2- 172.21.15.52:0/2794280198 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc6ac15f950 0x7fc6ac15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.079+0000 7fc6b2aca640 1 --2- 172.21.15.52:0/2794280198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6ac15e5f0 0x7fc6ac15a260 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.079+0000 7fc6b2aca640 1 --2- 172.21.15.52:0/2794280198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6ac0cd5f0 0x7fc6ac143f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.079+0000 7fc6b2aca640 1 -- 172.21.15.52:0/2794280198 >> 172.21.15.52:0/2794280198 conn(0x7fc6ac0b2380 msgr2=0x7fc6ac0cba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:36.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.079+0000 7fc6b2aca640 1 -- 172.21.15.52:0/2794280198 shutdown_connections 2024-05-04T21:20:36.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.079+0000 7fc6b2aca640 1 -- 172.21.15.52:0/2794280198 wait complete. 2024-05-04T21:20:36.100 INFO:tasks.ceph:Scrubbing osd.5 2024-05-04T21:20:36.101 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-04T21:20:36.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.227+0000 7fbf2bbaf640 1 Processor -- start 2024-05-04T21:20:36.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2bbaf640 1 -- start start 2024-05-04T21:20:36.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2bbaf640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf24144fc0 0x7fbf240ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2bbaf640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf240ced50 con 0x7fbf2406f560 2024-05-04T21:20:36.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2bbaf640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf2415fe10 con 0x7fbf241608f0 2024-05-04T21:20:36.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2bbaf640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fbf241453a0 con 0x7fbf24144fc0 2024-05-04T21:20:36.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2a14c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fbf2406f560 0x7fbf240ccac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51706/0 (socket says 172.21.15.52:51706) 2024-05-04T21:20:36.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2a14c640 1 -- 172.21.15.52:0/4190376110 learned_addr learned my addr 172.21.15.52:0/4190376110 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:36.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2994b640 1 --2- 172.21.15.52:0/4190376110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf24144fc0 0x7fbf240ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 -- 172.21.15.52:0/4190376110 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2066320839 0 0) 0x7fbf240ced50 con 0x7fbf2406f560 2024-05-04T21:20:36.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 -- 172.21.15.52:0/4190376110 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf10003530 con 0x7fbf2406f560 2024-05-04T21:20:36.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 -- 172.21.15.52:0/4190376110 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2365170703 0 0) 0x7fbf10003530 con 0x7fbf2406f560 2024-05-04T21:20:36.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 -- 172.21.15.52:0/4190376110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf24144fc0 msgr2=0x7fbf240ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:36.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 --2- 172.21.15.52:0/4190376110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf24144fc0 0x7fbf240ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 -- 172.21.15.52:0/4190376110 >> v1:172.21.15.111:6789/0 conn(0x7fbf241608f0 legacy=0x7fbf2406ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 -- 172.21.15.52:0/4190376110 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf241600f0 con 0x7fbf2406f560 2024-05-04T21:20:36.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 -- 172.21.15.52:0/4190376110 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7fbf1c001680 con 0x7fbf2406f560 2024-05-04T21:20:36.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 -- 172.21.15.52:0/4190376110 >> v1:172.21.15.52:6789/0 conn(0x7fbf2406f560 legacy=0x7fbf240ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 --2- 172.21.15.52:0/4190376110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf24144fc0 0x7fbf100039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 --2- 172.21.15.52:0/4190376110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf10003fc0 0x7fbf10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 --2- 172.21.15.52:0/4190376110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10004d50 0x7fbf100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 -- 172.21.15.52:0/4190376110 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1000bb90 con 0x7fbf10003fc0 2024-05-04T21:20:36.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 -- 172.21.15.52:0/4190376110 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1000bd40 con 0x7fbf10004d50 2024-05-04T21:20:36.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf28949640 1 -- 172.21.15.52:0/4190376110 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1000bf40 con 0x7fbf24144fc0 2024-05-04T21:20:36.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2914a640 1 --2- 172.21.15.52:0/4190376110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf10003fc0 0x7fbf10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2994b640 1 --2- 172.21.15.52:0/4190376110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf24144fc0 0x7fbf100039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2a14c640 1 --2- 172.21.15.52:0/4190376110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10004d50 0x7fbf100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2914a640 1 -- 172.21.15.52:0/4190376110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf24144fc0 msgr2=0x7fbf100039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:36.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2914a640 1 --2- 172.21.15.52:0/4190376110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf24144fc0 0x7fbf100039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2914a640 1 -- 172.21.15.52:0/4190376110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10004d50 msgr2=0x7fbf100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:36.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2914a640 1 --2- 172.21.15.52:0/4190376110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10004d50 0x7fbf100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2914a640 1 -- 172.21.15.52:0/4190376110 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf10003530 con 0x7fbf10003fc0 2024-05-04T21:20:36.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.231+0000 7fbf2a14c640 1 --2- 172.21.15.52:0/4190376110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10004d50 0x7fbf100051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:36.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2914a640 1 --2- 172.21.15.52:0/4190376110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf10003fc0 0x7fbf10004810 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fbf14001780 tx=0x7fbf1401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50a1d4081e843a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:36.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf28949640 1 -- 172.21.15.52:0/4190376110 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf140013a0 con 0x7fbf10003fc0 2024-05-04T21:20:36.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf28949640 1 -- 172.21.15.52:0/4190376110 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf140052f0 con 0x7fbf10003fc0 2024-05-04T21:20:36.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf28949640 1 -- 172.21.15.52:0/4190376110 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14003840 con 0x7fbf10003fc0 2024-05-04T21:20:36.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/4190376110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf10003fc0 msgr2=0x7fbf10004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2bbaf640 1 --2- 172.21.15.52:0/4190376110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf10003fc0 0x7fbf10004810 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fbf14001780 tx=0x7fbf1401c420 comp rx=0 tx=0).stop 2024-05-04T21:20:36.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2a14c640 1 -- 172.21.15.52:0/4190376110 reap_dead start 2024-05-04T21:20:36.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/4190376110 shutdown_connections 2024-05-04T21:20:36.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/4190376110 >> 172.21.15.52:0/4190376110 conn(0x7fbf24067760 msgr2=0x7fbf240b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:36.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/4190376110 shutdown_connections 2024-05-04T21:20:36.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/4190376110 wait complete. 2024-05-04T21:20:36.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2bbaf640 1 Processor -- start 2024-05-04T21:20:36.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2bbaf640 1 -- start start 2024-05-04T21:20:36.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2bbaf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10003fc0 0x7fbf2414c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2bbaf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf2414f0f0 0x7fbf2414dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2bbaf640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf24143d30 0x7fbf240adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2bbaf640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf240ce690 con 0x7fbf2414f0f0 2024-05-04T21:20:36.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2bbaf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbf240ce510 con 0x7fbf10003fc0 2024-05-04T21:20:36.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2bbaf640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fbf240ce810 con 0x7fbf24143d30 2024-05-04T21:20:36.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2914a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf2414f0f0 0x7fbf2414dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2914a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf2414f0f0 0x7fbf2414dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40666/0 (socket says 172.21.15.52:40666) 2024-05-04T21:20:36.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.235+0000 7fbf2914a640 1 -- 172.21.15.52:0/1438980300 learned_addr learned my addr 172.21.15.52:0/1438980300 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:36.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2994b640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10003fc0 0x7fbf2414c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2a14c640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf24143d30 0x7fbf240adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2914a640 1 -- 172.21.15.52:0/1438980300 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf24143d30 msgr2=0x7fbf240adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2914a640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf24143d30 0x7fbf240adee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2914a640 1 -- 172.21.15.52:0/1438980300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10003fc0 msgr2=0x7fbf2414c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2914a640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10003fc0 0x7fbf2414c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2914a640 1 -- 172.21.15.52:0/1438980300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2414f570 con 0x7fbf2414f0f0 2024-05-04T21:20:36.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2994b640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf10003fc0 0x7fbf2414c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:36.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2a14c640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf24143d30 0x7fbf240adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:36.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2914a640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf2414f0f0 0x7fbf2414dd40 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fbf14005040 tx=0x7fbf10005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:36.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf1a7fc640 1 -- 172.21.15.52:0/1438980300 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14007a40 con 0x7fbf2414f0f0 2024-05-04T21:20:36.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf1a7fc640 1 -- 172.21.15.52:0/1438980300 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf140bb070 con 0x7fbf2414f0f0 2024-05-04T21:20:36.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf1a7fc640 1 -- 172.21.15.52:0/1438980300 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14000ee0 con 0x7fbf2414f0f0 2024-05-04T21:20:36.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/1438980300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2414cca0 con 0x7fbf2414f0f0 2024-05-04T21:20:36.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/1438980300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2414d260 con 0x7fbf2414f0f0 2024-05-04T21:20:36.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/1438980300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf2414d920 con 0x7fbf2414f0f0 2024-05-04T21:20:36.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf1a7fc640 1 -- 172.21.15.52:0/1438980300 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbf140b4040 con 0x7fbf2414f0f0 2024-05-04T21:20:36.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf1a7fc640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbf08067a60 0x7fbf08069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf1a7fc640 1 -- 172.21.15.52:0/1438980300 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fbf141200e0 con 0x7fbf2414f0f0 2024-05-04T21:20:36.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2994b640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbf08067a60 0x7fbf08069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf1a7fc640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.111:6803/1318681540,v1:172.21.15.111:6807/1318681540] conn(0x7fbf0806d920 0x7fbf0806fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf1a7fc640 1 -- 172.21.15.52:0/1438980300 --> [v2:172.21.15.111:6803/1318681540,v1:172.21.15.111:6807/1318681540] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf14004e90 con 0x7fbf0806d920 2024-05-04T21:20:36.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf1a7fc640 1 -- 172.21.15.52:0/1438980300 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbf14120570 con 0x7fbf2414f0f0 2024-05-04T21:20:36.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2a14c640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.111:6803/1318681540,v1:172.21.15.111:6807/1318681540] conn(0x7fbf0806d920 0x7fbf0806fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2994b640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbf08067a60 0x7fbf08069f20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fbf20007440 tx=0x7fbf2000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:36.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.239+0000 7fbf2a14c640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.111:6803/1318681540,v1:172.21.15.111:6807/1318681540] conn(0x7fbf0806d920 0x7fbf0806fd40 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:36.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.251+0000 7fbf1a7fc640 1 -- 172.21.15.52:0/1438980300 <== osd.5 v2:172.21.15.111:6803/1318681540 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf14004e90 con 0x7fbf0806d920 2024-05-04T21:20:36.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.263+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/1438980300 --> [v2:172.21.15.111:6803/1318681540,v1:172.21.15.111:6807/1318681540] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbf24062e50 con 0x7fbf0806d920 2024-05-04T21:20:36.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.283+0000 7fbf1a7fc640 1 -- 172.21.15.52:0/1438980300 <== osd.5 v2:172.21.15.111:6803/1318681540 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbf24062e50 con 0x7fbf0806d920 2024-05-04T21:20:36.289 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-04T21:20:36.289 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-04T21:20:36.289 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-04T21:20:36.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.283+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/1438980300 >> [v2:172.21.15.111:6803/1318681540,v1:172.21.15.111:6807/1318681540] conn(0x7fbf0806d920 msgr2=0x7fbf0806fd40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.283+0000 7fbf2bbaf640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.111:6803/1318681540,v1:172.21.15.111:6807/1318681540] conn(0x7fbf0806d920 0x7fbf0806fd40 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.283+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/1438980300 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbf08067a60 msgr2=0x7fbf08069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.283+0000 7fbf2bbaf640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbf08067a60 0x7fbf08069f20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fbf20007440 tx=0x7fbf2000b040 comp rx=0 tx=0).stop 2024-05-04T21:20:36.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.283+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/1438980300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf2414f0f0 msgr2=0x7fbf2414dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.283+0000 7fbf2bbaf640 1 --2- 172.21.15.52:0/1438980300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf2414f0f0 0x7fbf2414dd40 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fbf14005040 tx=0x7fbf10005840 comp rx=0 tx=0).stop 2024-05-04T21:20:36.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.283+0000 7fbf2a14c640 1 -- 172.21.15.52:0/1438980300 reap_dead start 2024-05-04T21:20:36.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.283+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/1438980300 shutdown_connections 2024-05-04T21:20:36.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.283+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/1438980300 >> 172.21.15.52:0/1438980300 conn(0x7fbf24067760 msgr2=0x7fbf240af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:36.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.283+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/1438980300 shutdown_connections 2024-05-04T21:20:36.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.283+0000 7fbf2bbaf640 1 -- 172.21.15.52:0/1438980300 wait complete. 2024-05-04T21:20:36.310 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-04T21:20:36.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb394995640 1 Processor -- start 2024-05-04T21:20:36.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb394995640 1 -- start start 2024-05-04T21:20:36.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb394995640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3900aa0c0 0x7fb3900ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb394995640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3900aa550 0x7fb3900cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb394995640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb39006d1b0 con 0x7fb39015bf50 2024-05-04T21:20:36.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb394995640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb39015ca40 con 0x7fb3900aa0c0 2024-05-04T21:20:36.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb394995640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fb39006d490 con 0x7fb3900aa550 2024-05-04T21:20:36.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb38dd9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3900aa0c0 0x7fb3900ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb38dd9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3900aa0c0 0x7fb3900ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40678/0 (socket says 172.21.15.52:40678) 2024-05-04T21:20:36.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb38dd9b640 1 -- 172.21.15.52:0/4277125610 learned_addr learned my addr 172.21.15.52:0/4277125610 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:36.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb38ed9d640 1 --2- 172.21.15.52:0/4277125610 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3900aa550 0x7fb3900cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb38dd9b640 1 -- 172.21.15.52:0/4277125610 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3900aa550 msgr2=0x7fb3900cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:36.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb38dd9b640 1 --2- 172.21.15.52:0/4277125610 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3900aa550 0x7fb3900cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb38dd9b640 1 -- 172.21.15.52:0/4277125610 >> v1:172.21.15.111:6789/0 conn(0x7fb39015bf50 legacy=0x7fb39015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.439+0000 7fb38dd9b640 1 -- 172.21.15.52:0/4277125610 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39006d770 con 0x7fb3900aa0c0 2024-05-04T21:20:36.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb38dd9b640 1 --2- 172.21.15.52:0/4277125610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3900aa0c0 0x7fb3900ccb50 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fb380009870 tx=0x7fb38001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf090a7ad8fcc9b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:36.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb38d59a640 1 -- 172.21.15.52:0/4277125610 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3800a9610 con 0x7fb3900aa0c0 2024-05-04T21:20:36.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb38d59a640 1 -- 172.21.15.52:0/4277125610 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3800a97b0 con 0x7fb3900aa0c0 2024-05-04T21:20:36.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb38d59a640 1 -- 172.21.15.52:0/4277125610 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3800a9b80 con 0x7fb3900aa0c0 2024-05-04T21:20:36.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 -- 172.21.15.52:0/4277125610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3900aa0c0 msgr2=0x7fb3900ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 --2- 172.21.15.52:0/4277125610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3900aa0c0 0x7fb3900ccb50 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fb380009870 tx=0x7fb38001c530 comp rx=0 tx=0).stop 2024-05-04T21:20:36.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 -- 172.21.15.52:0/4277125610 shutdown_connections 2024-05-04T21:20:36.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 --2- 172.21.15.52:0/4277125610 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3900aa550 0x7fb3900cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 --2- 172.21.15.52:0/4277125610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3900aa0c0 0x7fb3900ccb50 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 -- 172.21.15.52:0/4277125610 >> 172.21.15.52:0/4277125610 conn(0x7fb3900b14a0 msgr2=0x7fb3900b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:36.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 -- 172.21.15.52:0/4277125610 shutdown_connections 2024-05-04T21:20:36.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 -- 172.21.15.52:0/4277125610 wait complete. 2024-05-04T21:20:36.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 Processor -- start 2024-05-04T21:20:36.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 -- start start 2024-05-04T21:20:36.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3900aa0c0 0x7fb3901f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3900aa550 0x7fb390152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb39015bf50 0x7fb390154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb39006ebe0 con 0x7fb3900aa0c0 2024-05-04T21:20:36.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb39006ea60 con 0x7fb3900aa550 2024-05-04T21:20:36.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.443+0000 7fb394995640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fb39006ed60 con 0x7fb39015bf50 2024-05-04T21:20:36.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb38e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3900aa0c0 0x7fb3901f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb38e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3900aa0c0 0x7fb3901f8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40680/0 (socket says 172.21.15.52:40680) 2024-05-04T21:20:36.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb38e59c640 1 -- 172.21.15.52:0/1957071110 learned_addr learned my addr 172.21.15.52:0/1957071110 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:36.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb38dd9b640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3900aa550 0x7fb390152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb38ed9d640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb39015bf50 0x7fb390154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb38e59c640 1 -- 172.21.15.52:0/1957071110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb39015bf50 msgr2=0x7fb390154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb38e59c640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb39015bf50 0x7fb390154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb38e59c640 1 -- 172.21.15.52:0/1957071110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3900aa550 msgr2=0x7fb390152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb38e59c640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3900aa550 0x7fb390152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb38e59c640 1 -- 172.21.15.52:0/1957071110 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3901f8e10 con 0x7fb3900aa0c0 2024-05-04T21:20:36.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb38ed9d640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb39015bf50 0x7fb390154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T21:20:36.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb38e59c640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3900aa0c0 0x7fb3901f8710 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fb37800c370 tx=0x7fb37800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:36.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb386ffd640 1 -- 172.21.15.52:0/1957071110 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3780052d0 con 0x7fb3900aa0c0 2024-05-04T21:20:36.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb386ffd640 1 -- 172.21.15.52:0/1957071110 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb378005470 con 0x7fb3900aa0c0 2024-05-04T21:20:36.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb386ffd640 1 -- 172.21.15.52:0/1957071110 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378007260 con 0x7fb3900aa0c0 2024-05-04T21:20:36.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb394995640 1 -- 172.21.15.52:0/1957071110 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3901f90f0 con 0x7fb3900aa0c0 2024-05-04T21:20:36.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb394995640 1 -- 172.21.15.52:0/1957071110 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb390153890 con 0x7fb3900aa0c0 2024-05-04T21:20:36.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb394995640 1 -- 172.21.15.52:0/1957071110 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb390062e50 con 0x7fb3900aa0c0 2024-05-04T21:20:36.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb386ffd640 1 -- 172.21.15.52:0/1957071110 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb378007400 con 0x7fb3900aa0c0 2024-05-04T21:20:36.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb386ffd640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3700678f0 0x7fb370069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb386ffd640 1 -- 172.21.15.52:0/1957071110 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fb378085520 con 0x7fb3900aa0c0 2024-05-04T21:20:36.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb38dd9b640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3700678f0 0x7fb370069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.447+0000 7fb38dd9b640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3700678f0 0x7fb370069db0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb3800ae000 tx=0x7fb3800b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:36.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.451+0000 7fb386ffd640 1 -- 172.21.15.52:0/1957071110 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb378055df0 con 0x7fb3900aa0c0 2024-05-04T21:20:36.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.607+0000 7fb394995640 1 -- 172.21.15.52:0/1957071110 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fb390155490 con 0x7fb3700678f0 2024-05-04T21:20:36.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.607+0000 7fb386ffd640 1 -- 172.21.15.52:0/1957071110 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb3800a9490 con 0x7fb3700678f0 2024-05-04T21:20:36.611 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 5 to deep-scrub 2024-05-04T21:20:36.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.607+0000 7fb394995640 1 -- 172.21.15.52:0/1957071110 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3700678f0 msgr2=0x7fb370069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.607+0000 7fb394995640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3700678f0 0x7fb370069db0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb3800ae000 tx=0x7fb3800b3040 comp rx=0 tx=0).stop 2024-05-04T21:20:36.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.607+0000 7fb394995640 1 -- 172.21.15.52:0/1957071110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3900aa0c0 msgr2=0x7fb3901f8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.607+0000 7fb394995640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3900aa0c0 0x7fb3901f8710 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fb37800c370 tx=0x7fb37800c830 comp rx=0 tx=0).stop 2024-05-04T21:20:36.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.607+0000 7fb394995640 1 -- 172.21.15.52:0/1957071110 shutdown_connections 2024-05-04T21:20:36.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.607+0000 7fb394995640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fb3700678f0 0x7fb370069db0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.607+0000 7fb394995640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb39015bf50 0x7fb390154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.607+0000 7fb394995640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3900aa550 0x7fb390152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.607+0000 7fb394995640 1 --2- 172.21.15.52:0/1957071110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3900aa0c0 0x7fb3901f8710 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.607+0000 7fb394995640 1 -- 172.21.15.52:0/1957071110 >> 172.21.15.52:0/1957071110 conn(0x7fb3900b14a0 msgr2=0x7fb3901597a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:36.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.607+0000 7fb394995640 1 -- 172.21.15.52:0/1957071110 shutdown_connections 2024-05-04T21:20:36.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.607+0000 7fb394995640 1 -- 172.21.15.52:0/1957071110 wait complete. 2024-05-04T21:20:36.629 INFO:tasks.ceph:Scrubbing osd.6 2024-05-04T21:20:36.629 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-04T21:20:36.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.755+0000 7f7942fcc640 1 Processor -- start 2024-05-04T21:20:36.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.755+0000 7f7942fcc640 1 -- start start 2024-05-04T21:20:36.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.759+0000 7f7942fcc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f793c0728d0 0x7f793c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.759+0000 7f7942fcc640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f793c0cf250 0x7f793c0ae530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.759+0000 7f7942fcc640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f793c1450d0 con 0x7f793c144cf0 2024-05-04T21:20:36.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.759+0000 7f7942fcc640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f793c0ae0c0 con 0x7f793c0728d0 2024-05-04T21:20:36.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.759+0000 7f7942fcc640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f793c0d12a0 con 0x7f793c0cf250 2024-05-04T21:20:36.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.759+0000 7f793bfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f793c0728d0 0x7f793c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.759+0000 7f7941569640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f793c0cf250 0x7f793c0ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.759+0000 7f793bfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f793c0728d0 0x7f793c0ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.52:47836/0 (socket says 172.21.15.52:47836) 2024-05-04T21:20:36.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.759+0000 7f793bfff640 1 -- 172.21.15.52:0/1615121066 learned_addr learned my addr 172.21.15.52:0/1615121066 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:36.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.759+0000 7f7941569640 1 -- 172.21.15.52:0/1615121066 >> v2:172.21.15.111:3300/0 conn(0x7f793c0728d0 msgr2=0x7f793c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:36.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.759+0000 7f7941569640 1 --2- 172.21.15.52:0/1615121066 >> v2:172.21.15.111:3300/0 conn(0x7f793c0728d0 0x7f793c0ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.759+0000 7f7941569640 1 -- 172.21.15.52:0/1615121066 >> v1:172.21.15.111:6789/0 conn(0x7f793c144cf0 legacy=0x7f793c0d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.759+0000 7f7941569640 1 -- 172.21.15.52:0/1615121066 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c0d1500 con 0x7f793c0cf250 2024-05-04T21:20:36.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7941569640 1 --2- 172.21.15.52:0/1615121066 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f793c0cf250 0x7f793c0ae530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7934009d90 tx=0x7f793401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=929a76a1eb5d92c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:36.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f793b7fe640 1 -- 172.21.15.52:0/1615121066 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79340b2070 con 0x7f793c0cf250 2024-05-04T21:20:36.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f793b7fe640 1 -- 172.21.15.52:0/1615121066 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79340a5830 con 0x7f793c0cf250 2024-05-04T21:20:36.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f793b7fe640 1 -- 172.21.15.52:0/1615121066 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79340a5c00 con 0x7f793c0cf250 2024-05-04T21:20:36.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 -- 172.21.15.52:0/1615121066 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f793c0cf250 msgr2=0x7f793c0ae530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 --2- 172.21.15.52:0/1615121066 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f793c0cf250 0x7f793c0ae530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7934009d90 tx=0x7f793401c970 comp rx=0 tx=0).stop 2024-05-04T21:20:36.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 -- 172.21.15.52:0/1615121066 shutdown_connections 2024-05-04T21:20:36.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 --2- 172.21.15.52:0/1615121066 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f793c0cf250 0x7f793c0ae530 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 --2- 172.21.15.52:0/1615121066 >> v2:172.21.15.111:3300/0 conn(0x7f793c0728d0 0x7f793c0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 -- 172.21.15.52:0/1615121066 >> 172.21.15.52:0/1615121066 conn(0x7f793c06ee80 msgr2=0x7f793c06f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:36.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 -- 172.21.15.52:0/1615121066 shutdown_connections 2024-05-04T21:20:36.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 -- 172.21.15.52:0/1615121066 wait complete. 2024-05-04T21:20:36.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 Processor -- start 2024-05-04T21:20:36.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 -- start start 2024-05-04T21:20:36.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f793c0728d0 0x7f793c140a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f793c0cf250 0x7f793c1687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f793c144cf0 0x7f793c1433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f793c0af400 con 0x7f793c144cf0 2024-05-04T21:20:36.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f793c0af280 con 0x7f793c0cf250 2024-05-04T21:20:36.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.763+0000 7f7942fcc640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f793c0af580 con 0x7f793c0728d0 2024-05-04T21:20:36.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7941569640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f793c144cf0 0x7f793c1433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7941569640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f793c144cf0 0x7f793c1433e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40686/0 (socket says 172.21.15.52:40686) 2024-05-04T21:20:36.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7941569640 1 -- 172.21.15.52:0/364381316 learned_addr learned my addr 172.21.15.52:0/364381316 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:36.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f793bfff640 1 --2- 172.21.15.52:0/364381316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f793c0cf250 0x7f793c1687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7940d68640 1 --2- 172.21.15.52:0/364381316 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f793c0728d0 0x7f793c140a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7941569640 1 -- 172.21.15.52:0/364381316 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f793c0728d0 msgr2=0x7f793c140a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7941569640 1 --2- 172.21.15.52:0/364381316 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f793c0728d0 0x7f793c140a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7941569640 1 -- 172.21.15.52:0/364381316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f793c0cf250 msgr2=0x7f793c1687f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7941569640 1 --2- 172.21.15.52:0/364381316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f793c0cf250 0x7f793c1687f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7941569640 1 -- 172.21.15.52:0/364381316 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c15b750 con 0x7f793c144cf0 2024-05-04T21:20:36.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7941569640 1 --2- 172.21.15.52:0/364381316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f793c144cf0 0x7f793c1433e0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f79340b0040 tx=0x7f7934019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:36.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f79397fa640 1 -- 172.21.15.52:0/364381316 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79340b2050 con 0x7f793c144cf0 2024-05-04T21:20:36.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f79397fa640 1 -- 172.21.15.52:0/364381316 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79340b3040 con 0x7f793c144cf0 2024-05-04T21:20:36.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f79397fa640 1 -- 172.21.15.52:0/364381316 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79340ad650 con 0x7f793c144cf0 2024-05-04T21:20:36.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7942fcc640 1 -- 172.21.15.52:0/364381316 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793c15b9f0 con 0x7f793c144cf0 2024-05-04T21:20:36.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7942fcc640 1 -- 172.21.15.52:0/364381316 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f793c15bfb0 con 0x7f793c144cf0 2024-05-04T21:20:36.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7942fcc640 1 -- 172.21.15.52:0/364381316 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f793c062e50 con 0x7f793c144cf0 2024-05-04T21:20:36.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f79397fa640 1 -- 172.21.15.52:0/364381316 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f79340c0020 con 0x7f793c144cf0 2024-05-04T21:20:36.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f79397fa640 1 --2- 172.21.15.52:0/364381316 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f79200678c0 0x7f7920069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f79397fa640 1 -- 172.21.15.52:0/364381316 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f7934120d40 con 0x7f793c144cf0 2024-05-04T21:20:36.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7940d68640 1 --2- 172.21.15.52:0/364381316 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f79200678c0 0x7f7920069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f79397fa640 1 --2- 172.21.15.52:0/364381316 >> [v2:172.21.15.111:6800/1985934042,v1:172.21.15.111:6801/1985934042] conn(0x7f792006d780 0x7f792006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f79397fa640 1 -- 172.21.15.52:0/364381316 --> [v2:172.21.15.111:6800/1985934042,v1:172.21.15.111:6801/1985934042] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7934006ba0 con 0x7f792006d780 2024-05-04T21:20:36.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f79397fa640 1 -- 172.21.15.52:0/364381316 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f79341210f0 con 0x7f793c144cf0 2024-05-04T21:20:36.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f793bfff640 1 --2- 172.21.15.52:0/364381316 >> [v2:172.21.15.111:6800/1985934042,v1:172.21.15.111:6801/1985934042] conn(0x7f792006d780 0x7f792006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.767+0000 7f7940d68640 1 --2- 172.21.15.52:0/364381316 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f79200678c0 0x7f7920069d80 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7930001770 tx=0x7f7930009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:36.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.771+0000 7f793bfff640 1 --2- 172.21.15.52:0/364381316 >> [v2:172.21.15.111:6800/1985934042,v1:172.21.15.111:6801/1985934042] conn(0x7f792006d780 0x7f792006fba0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:36.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.779+0000 7f79397fa640 1 -- 172.21.15.52:0/364381316 <== osd.6 v2:172.21.15.111:6800/1985934042 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7934006ba0 con 0x7f792006d780 2024-05-04T21:20:36.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.791+0000 7f7942fcc640 1 -- 172.21.15.52:0/364381316 --> [v2:172.21.15.111:6800/1985934042,v1:172.21.15.111:6801/1985934042] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f793c1668a0 con 0x7f792006d780 2024-05-04T21:20:36.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.803+0000 7f79397fa640 1 -- 172.21.15.52:0/364381316 <== osd.6 v2:172.21.15.111:6800/1985934042 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f793c1668a0 con 0x7f792006d780 2024-05-04T21:20:36.806 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-04T21:20:36.806 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-04T21:20:36.807 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-04T21:20:36.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.803+0000 7f7942fcc640 1 -- 172.21.15.52:0/364381316 >> [v2:172.21.15.111:6800/1985934042,v1:172.21.15.111:6801/1985934042] conn(0x7f792006d780 msgr2=0x7f792006fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.803+0000 7f7942fcc640 1 --2- 172.21.15.52:0/364381316 >> [v2:172.21.15.111:6800/1985934042,v1:172.21.15.111:6801/1985934042] conn(0x7f792006d780 0x7f792006fba0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.803+0000 7f7942fcc640 1 -- 172.21.15.52:0/364381316 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f79200678c0 msgr2=0x7f7920069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.803+0000 7f7942fcc640 1 --2- 172.21.15.52:0/364381316 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f79200678c0 0x7f7920069d80 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7930001770 tx=0x7f7930009040 comp rx=0 tx=0).stop 2024-05-04T21:20:36.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.803+0000 7f7942fcc640 1 -- 172.21.15.52:0/364381316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f793c144cf0 msgr2=0x7f793c1433e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.803+0000 7f7942fcc640 1 --2- 172.21.15.52:0/364381316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f793c144cf0 0x7f793c1433e0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f79340b0040 tx=0x7f7934019710 comp rx=0 tx=0).stop 2024-05-04T21:20:36.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.803+0000 7f7941569640 1 -- 172.21.15.52:0/364381316 reap_dead start 2024-05-04T21:20:36.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.803+0000 7f7942fcc640 1 -- 172.21.15.52:0/364381316 shutdown_connections 2024-05-04T21:20:36.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.803+0000 7f7942fcc640 1 -- 172.21.15.52:0/364381316 >> 172.21.15.52:0/364381316 conn(0x7f793c06ee80 msgr2=0x7f793c0af910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:36.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.803+0000 7f7942fcc640 1 -- 172.21.15.52:0/364381316 shutdown_connections 2024-05-04T21:20:36.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.803+0000 7f7942fcc640 1 -- 172.21.15.52:0/364381316 wait complete. 2024-05-04T21:20:36.824 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-04T21:20:36.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb2539c640 1 Processor -- start 2024-05-04T21:20:36.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb2539c640 1 -- start start 2024-05-04T21:20:36.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb2539c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fcb200aa0c0 0x7fcb200ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb2539c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb200aa550 0x7fcb200cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb2539c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb2006d1b0 con 0x7fcb2015bf50 2024-05-04T21:20:36.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb2539c640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fcb2015ca40 con 0x7fcb200aa0c0 2024-05-04T21:20:36.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb2539c640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fcb2006d490 con 0x7fcb200aa550 2024-05-04T21:20:36.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb167fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fcb200aa0c0 0x7fcb200ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb167fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fcb200aa0c0 0x7fcb200ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40700/0 (socket says 172.21.15.52:40700) 2024-05-04T21:20:36.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb167fc640 1 -- 172.21.15.52:0/3557301107 learned_addr learned my addr 172.21.15.52:0/3557301107 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:36.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb1f7fe640 1 --2- 172.21.15.52:0/3557301107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb200aa550 0x7fcb200cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb167fc640 1 -- 172.21.15.52:0/3557301107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb200aa550 msgr2=0x7fcb200cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:36.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb167fc640 1 --2- 172.21.15.52:0/3557301107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb200aa550 0x7fcb200cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb167fc640 1 -- 172.21.15.52:0/3557301107 >> v1:172.21.15.111:6789/0 conn(0x7fcb2015bf50 legacy=0x7fcb2015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:36.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.947+0000 7fcb167fc640 1 -- 172.21.15.52:0/3557301107 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2006d770 con 0x7fcb200aa0c0 2024-05-04T21:20:36.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.951+0000 7fcb167fc640 1 --2- 172.21.15.52:0/3557301107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb200aa0c0 0x7fcb200ccb50 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009870 tx=0x7fcb0801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7570d76a30db84ff server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:36.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.951+0000 7fcb1e7fc640 1 -- 172.21.15.52:0/3557301107 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb080a9610 con 0x7fcb200aa0c0 2024-05-04T21:20:36.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.951+0000 7fcb1e7fc640 1 -- 172.21.15.52:0/3557301107 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb080a97b0 con 0x7fcb200aa0c0 2024-05-04T21:20:36.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.951+0000 7fcb1e7fc640 1 -- 172.21.15.52:0/3557301107 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb080a9b80 con 0x7fcb200aa0c0 2024-05-04T21:20:36.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.951+0000 7fcb2539c640 1 -- 172.21.15.52:0/3557301107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb200aa0c0 msgr2=0x7fcb200ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.951+0000 7fcb2539c640 1 --2- 172.21.15.52:0/3557301107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb200aa0c0 0x7fcb200ccb50 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009870 tx=0x7fcb0801c530 comp rx=0 tx=0).stop 2024-05-04T21:20:36.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.951+0000 7fcb2539c640 1 -- 172.21.15.52:0/3557301107 shutdown_connections 2024-05-04T21:20:36.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.951+0000 7fcb2539c640 1 --2- 172.21.15.52:0/3557301107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb200aa550 0x7fcb200cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.951+0000 7fcb2539c640 1 --2- 172.21.15.52:0/3557301107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb200aa0c0 0x7fcb200ccb50 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.951+0000 7fcb2539c640 1 -- 172.21.15.52:0/3557301107 >> 172.21.15.52:0/3557301107 conn(0x7fcb200b14a0 msgr2=0x7fcb200b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:36.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.951+0000 7fcb2539c640 1 -- 172.21.15.52:0/3557301107 shutdown_connections 2024-05-04T21:20:36.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.951+0000 7fcb2539c640 1 -- 172.21.15.52:0/3557301107 wait complete. 2024-05-04T21:20:36.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.951+0000 7fcb2539c640 1 Processor -- start 2024-05-04T21:20:36.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.951+0000 7fcb2539c640 1 -- start start 2024-05-04T21:20:36.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb2539c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb200aa0c0 0x7fcb20148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb2539c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb200aa550 0x7fcb201511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb2539c640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2015bf50 0x7fcb20147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb2539c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2006ebe0 con 0x7fcb200aa550 2024-05-04T21:20:36.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb2539c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2006ea60 con 0x7fcb200aa0c0 2024-05-04T21:20:36.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb2539c640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fcb2006ed60 con 0x7fcb2015bf50 2024-05-04T21:20:36.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb167fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb200aa550 0x7fcb201511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb167fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb200aa550 0x7fcb201511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40706/0 (socket says 172.21.15.52:40706) 2024-05-04T21:20:36.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb167fc640 1 -- 172.21.15.52:0/3266341661 learned_addr learned my addr 172.21.15.52:0/3266341661 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:36.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb1effd640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb200aa0c0 0x7fcb20148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb1f7fe640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2015bf50 0x7fcb20147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb167fc640 1 -- 172.21.15.52:0/3266341661 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2015bf50 msgr2=0x7fcb20147820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb167fc640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2015bf50 0x7fcb20147820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb167fc640 1 -- 172.21.15.52:0/3266341661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb200aa0c0 msgr2=0x7fcb20148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb167fc640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb200aa0c0 0x7fcb20148d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb167fc640 1 -- 172.21.15.52:0/3266341661 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb201495d0 con 0x7fcb200aa550 2024-05-04T21:20:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb1effd640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb200aa0c0 0x7fcb20148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb167fc640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb200aa550 0x7fcb201511c0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fcb080ab040 tx=0x7fcb0801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb17fff640 1 -- 172.21.15.52:0/3266341661 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb080bc650 con 0x7fcb200aa550 2024-05-04T21:20:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb17fff640 1 -- 172.21.15.52:0/3266341661 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb080bc810 con 0x7fcb200aa550 2024-05-04T21:20:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb2539c640 1 -- 172.21.15.52:0/3266341661 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2014e460 con 0x7fcb200aa550 2024-05-04T21:20:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb17fff640 1 -- 172.21.15.52:0/3266341661 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb080bcc00 con 0x7fcb200aa550 2024-05-04T21:20:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb2539c640 1 -- 172.21.15.52:0/3266341661 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2014eca0 con 0x7fcb200aa550 2024-05-04T21:20:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb2539c640 1 -- 172.21.15.52:0/3266341661 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb20062e50 con 0x7fcb200aa550 2024-05-04T21:20:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb17fff640 1 -- 172.21.15.52:0/3266341661 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcb080c53f0 con 0x7fcb200aa550 2024-05-04T21:20:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb17fff640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcb000678c0 0x7fcb00069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb17fff640 1 -- 172.21.15.52:0/3266341661 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fcb081202c0 con 0x7fcb200aa550 2024-05-04T21:20:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.955+0000 7fcb1effd640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcb000678c0 0x7fcb00069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.959+0000 7fcb1effd640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcb000678c0 0x7fcb00069d80 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fcb2014b2d0 tx=0x7fcb10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:36.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:36.959+0000 7fcb17fff640 1 -- 172.21.15.52:0/3266341661 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcb2014e460 con 0x7fcb200aa550 2024-05-04T21:20:37.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.127+0000 7fcb2539c640 1 -- 172.21.15.52:0/3266341661 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fcb20075320 con 0x7fcb000678c0 2024-05-04T21:20:37.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.127+0000 7fcb17fff640 1 -- 172.21.15.52:0/3266341661 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcb20075320 con 0x7fcb000678c0 2024-05-04T21:20:37.133 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 6 to deep-scrub 2024-05-04T21:20:37.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.131+0000 7fcb2539c640 1 -- 172.21.15.52:0/3266341661 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcb000678c0 msgr2=0x7fcb00069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.131+0000 7fcb2539c640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcb000678c0 0x7fcb00069d80 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fcb2014b2d0 tx=0x7fcb10009040 comp rx=0 tx=0).stop 2024-05-04T21:20:37.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.131+0000 7fcb2539c640 1 -- 172.21.15.52:0/3266341661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb200aa550 msgr2=0x7fcb201511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.131+0000 7fcb2539c640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb200aa550 0x7fcb201511c0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fcb080ab040 tx=0x7fcb0801ad70 comp rx=0 tx=0).stop 2024-05-04T21:20:37.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.131+0000 7fcb2539c640 1 -- 172.21.15.52:0/3266341661 shutdown_connections 2024-05-04T21:20:37.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.131+0000 7fcb2539c640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fcb000678c0 0x7fcb00069d80 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.131+0000 7fcb2539c640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2015bf50 0x7fcb20147820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.131+0000 7fcb2539c640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb200aa550 0x7fcb201511c0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.131+0000 7fcb2539c640 1 --2- 172.21.15.52:0/3266341661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb200aa0c0 0x7fcb20148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.131+0000 7fcb2539c640 1 -- 172.21.15.52:0/3266341661 >> 172.21.15.52:0/3266341661 conn(0x7fcb200b14a0 msgr2=0x7fcb2015c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:37.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.131+0000 7fcb2539c640 1 -- 172.21.15.52:0/3266341661 shutdown_connections 2024-05-04T21:20:37.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.131+0000 7fcb2539c640 1 -- 172.21.15.52:0/3266341661 wait complete. 2024-05-04T21:20:37.153 INFO:tasks.ceph:Scrubbing osd.7 2024-05-04T21:20:37.154 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-04T21:20:37.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.275+0000 7fbc33a45640 1 Processor -- start 2024-05-04T21:20:37.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.275+0000 7fbc33a45640 1 -- start start 2024-05-04T21:20:37.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.275+0000 7fbc33a45640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbc2c1553d0 0x7fbc2c0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.275+0000 7fbc33a45640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc2c0cdeb0 con 0x7fbc2c15c150 2024-05-04T21:20:37.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.275+0000 7fbc33a45640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc2c0cd2a0 con 0x7fbc2c0cfa70 2024-05-04T21:20:37.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.275+0000 7fbc33a45640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fbc2c0cd580 con 0x7fbc2c1553d0 2024-05-04T21:20:37.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.275+0000 7fbc30fe0640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fbc2c15c150 0x7fbc2c0cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51718/0 (socket says 172.21.15.52:51718) 2024-05-04T21:20:37.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.275+0000 7fbc30fe0640 1 -- 172.21.15.52:0/1791698990 learned_addr learned my addr 172.21.15.52:0/1791698990 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.275+0000 7fbc317e1640 1 --2- 172.21.15.52:0/1791698990 >> v2:172.21.15.111:3300/0 conn(0x7fbc2c1553d0 0x7fbc2c0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.275+0000 7fbc23fff640 1 -- 172.21.15.52:0/1791698990 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4092541741 0 0) 0x7fbc2c0cdeb0 con 0x7fbc2c15c150 2024-05-04T21:20:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.275+0000 7fbc23fff640 1 -- 172.21.15.52:0/1791698990 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc18003530 con 0x7fbc2c15c150 2024-05-04T21:20:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 -- 172.21.15.52:0/1791698990 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2693723526 0 0) 0x7fbc18003530 con 0x7fbc2c15c150 2024-05-04T21:20:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 -- 172.21.15.52:0/1791698990 >> v2:172.21.15.111:3300/0 conn(0x7fbc2c1553d0 msgr2=0x7fbc2c0cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 --2- 172.21.15.52:0/1791698990 >> v2:172.21.15.111:3300/0 conn(0x7fbc2c1553d0 0x7fbc2c0cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 -- 172.21.15.52:0/1791698990 >> v1:172.21.15.111:6789/0 conn(0x7fbc2c0cfa70 legacy=0x7fbc2c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 -- 172.21.15.52:0/1791698990 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c0cd860 con 0x7fbc2c15c150 2024-05-04T21:20:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 -- 172.21.15.52:0/1791698990 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7fbc1c001680 con 0x7fbc2c15c150 2024-05-04T21:20:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 -- 172.21.15.52:0/1791698990 >> v1:172.21.15.52:6789/0 conn(0x7fbc2c15c150 legacy=0x7fbc2c0cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 --2- 172.21.15.52:0/1791698990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc180039c0 0x7fbc18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 --2- 172.21.15.52:0/1791698990 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc18004810 0x7fbc18004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 --2- 172.21.15.52:0/1791698990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc180051d0 0x7fbc1800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 -- 172.21.15.52:0/1791698990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c0cd580 con 0x7fbc180039c0 2024-05-04T21:20:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 -- 172.21.15.52:0/1791698990 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1800c070 con 0x7fbc180051d0 2024-05-04T21:20:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 -- 172.21.15.52:0/1791698990 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1800c270 con 0x7fbc18004810 2024-05-04T21:20:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc317e1640 1 --2- 172.21.15.52:0/1791698990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc180039c0 0x7fbc18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc30fe0640 1 --2- 172.21.15.52:0/1791698990 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc18004810 0x7fbc18004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc317e1640 1 -- 172.21.15.52:0/1791698990 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc18004810 msgr2=0x7fbc18004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc317e1640 1 --2- 172.21.15.52:0/1791698990 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc18004810 0x7fbc18004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc317e1640 1 -- 172.21.15.52:0/1791698990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc180051d0 msgr2=0x7fbc1800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc317e1640 1 --2- 172.21.15.52:0/1791698990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc180051d0 0x7fbc1800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc317e1640 1 -- 172.21.15.52:0/1791698990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc18003530 con 0x7fbc180039c0 2024-05-04T21:20:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc31fe2640 1 -- 172.21.15.52:0/1791698990 reap_dead start 2024-05-04T21:20:37.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc317e1640 1 --2- 172.21.15.52:0/1791698990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc180039c0 0x7fbc18003de0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009ae0 tx=0x7fbc2801ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c712e754b7da789d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:37.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 -- 172.21.15.52:0/1791698990 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc280096b0 con 0x7fbc180039c0 2024-05-04T21:20:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 -- 172.21.15.52:0/1791698990 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc280a64b0 con 0x7fbc180039c0 2024-05-04T21:20:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc23fff640 1 -- 172.21.15.52:0/1791698990 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc280a68a0 con 0x7fbc180039c0 2024-05-04T21:20:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc33a45640 1 -- 172.21.15.52:0/1791698990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc180039c0 msgr2=0x7fbc18003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc33a45640 1 --2- 172.21.15.52:0/1791698990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc180039c0 0x7fbc18003de0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009ae0 tx=0x7fbc2801ca80 comp rx=0 tx=0).stop 2024-05-04T21:20:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc33a45640 1 -- 172.21.15.52:0/1791698990 shutdown_connections 2024-05-04T21:20:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc33a45640 1 --2- 172.21.15.52:0/1791698990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc180039c0 0x7fbc18003de0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc33a45640 1 -- 172.21.15.52:0/1791698990 >> 172.21.15.52:0/1791698990 conn(0x7fbc2c067760 msgr2=0x7fbc2c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc33a45640 1 -- 172.21.15.52:0/1791698990 shutdown_connections 2024-05-04T21:20:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.279+0000 7fbc33a45640 1 -- 172.21.15.52:0/1791698990 wait complete. 2024-05-04T21:20:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc33a45640 1 Processor -- start 2024-05-04T21:20:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc33a45640 1 -- start start 2024-05-04T21:20:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc33a45640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc180039c0 0x7fbc2c14ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc33a45640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc2c0b01c0 0x7fbc2c1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc33a45640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc2c157500 0x7fbc2c157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc33a45640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c15eb50 con 0x7fbc2c157500 2024-05-04T21:20:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc33a45640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c15e9d0 con 0x7fbc2c0b01c0 2024-05-04T21:20:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc33a45640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fbc2c15ecd0 con 0x7fbc180039c0 2024-05-04T21:20:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc31fe2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc2c157500 0x7fbc2c157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc31fe2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc2c157500 0x7fbc2c157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40728/0 (socket says 172.21.15.52:40728) 2024-05-04T21:20:37.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc31fe2640 1 -- 172.21.15.52:0/127587146 learned_addr learned my addr 172.21.15.52:0/127587146 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:37.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc317e1640 1 --2- 172.21.15.52:0/127587146 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc180039c0 0x7fbc2c14ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc30fe0640 1 --2- 172.21.15.52:0/127587146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc2c0b01c0 0x7fbc2c1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc31fe2640 1 -- 172.21.15.52:0/127587146 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc180039c0 msgr2=0x7fbc2c14ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc31fe2640 1 --2- 172.21.15.52:0/127587146 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc180039c0 0x7fbc2c14ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc31fe2640 1 -- 172.21.15.52:0/127587146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc2c0b01c0 msgr2=0x7fbc2c1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc31fe2640 1 --2- 172.21.15.52:0/127587146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc2c0b01c0 0x7fbc2c1559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc31fe2640 1 -- 172.21.15.52:0/127587146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c155f00 con 0x7fbc2c157500 2024-05-04T21:20:37.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc31fe2640 1 --2- 172.21.15.52:0/127587146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc2c157500 0x7fbc2c157980 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c158fd0 tx=0x7fbc2400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:37.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc21ffb640 1 -- 172.21.15.52:0/127587146 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24007070 con 0x7fbc2c157500 2024-05-04T21:20:37.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc21ffb640 1 -- 172.21.15.52:0/127587146 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc24007210 con 0x7fbc2c157500 2024-05-04T21:20:37.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc33a45640 1 -- 172.21.15.52:0/127587146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2c156d20 con 0x7fbc2c157500 2024-05-04T21:20:37.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc21ffb640 1 -- 172.21.15.52:0/127587146 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24007600 con 0x7fbc2c157500 2024-05-04T21:20:37.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc33a45640 1 -- 172.21.15.52:0/127587146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2c156480 con 0x7fbc2c157500 2024-05-04T21:20:37.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.283+0000 7fbc33a45640 1 -- 172.21.15.52:0/127587146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbc2c062e50 con 0x7fbc2c157500 2024-05-04T21:20:37.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.287+0000 7fbc21ffb640 1 -- 172.21.15.52:0/127587146 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbc24011070 con 0x7fbc2c157500 2024-05-04T21:20:37.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.287+0000 7fbc21ffb640 1 --2- 172.21.15.52:0/127587146 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbc100678f0 0x7fbc10069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.287+0000 7fbc21ffb640 1 -- 172.21.15.52:0/127587146 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fbc24085010 con 0x7fbc2c157500 2024-05-04T21:20:37.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.287+0000 7fbc317e1640 1 --2- 172.21.15.52:0/127587146 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbc100678f0 0x7fbc10069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.287+0000 7fbc21ffb640 1 --2- 172.21.15.52:0/127587146 >> [v2:172.21.15.111:6804/3267577455,v1:172.21.15.111:6806/3267577455] conn(0x7fbc1006d7b0 0x7fbc1006fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.287+0000 7fbc21ffb640 1 -- 172.21.15.52:0/127587146 --> [v2:172.21.15.111:6804/3267577455,v1:172.21.15.111:6806/3267577455] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbc2401eec0 con 0x7fbc1006d7b0 2024-05-04T21:20:37.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.287+0000 7fbc21ffb640 1 -- 172.21.15.52:0/127587146 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbc240854a0 con 0x7fbc2c157500 2024-05-04T21:20:37.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.287+0000 7fbc30fe0640 1 --2- 172.21.15.52:0/127587146 >> [v2:172.21.15.111:6804/3267577455,v1:172.21.15.111:6806/3267577455] conn(0x7fbc1006d7b0 0x7fbc1006fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.287+0000 7fbc317e1640 1 --2- 172.21.15.52:0/127587146 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbc100678f0 0x7fbc10069db0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fbc280af000 tx=0x7fbc28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:37.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.287+0000 7fbc30fe0640 1 --2- 172.21.15.52:0/127587146 >> [v2:172.21.15.111:6804/3267577455,v1:172.21.15.111:6806/3267577455] conn(0x7fbc1006d7b0 0x7fbc1006fbd0 crc :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:37.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.303+0000 7fbc21ffb640 1 -- 172.21.15.52:0/127587146 <== osd.7 v2:172.21.15.111:6804/3267577455 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbc2401eec0 con 0x7fbc1006d7b0 2024-05-04T21:20:37.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.315+0000 7fbc33a45640 1 -- 172.21.15.52:0/127587146 --> [v2:172.21.15.111:6804/3267577455,v1:172.21.15.111:6806/3267577455] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbc2c06d6e0 con 0x7fbc1006d7b0 2024-05-04T21:20:37.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.327+0000 7fbc21ffb640 1 -- 172.21.15.52:0/127587146 <== osd.7 v2:172.21.15.111:6804/3267577455 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbc2c06d6e0 con 0x7fbc1006d7b0 2024-05-04T21:20:37.332 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-04T21:20:37.332 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-04T21:20:37.332 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-04T21:20:37.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.327+0000 7fbc33a45640 1 -- 172.21.15.52:0/127587146 >> [v2:172.21.15.111:6804/3267577455,v1:172.21.15.111:6806/3267577455] conn(0x7fbc1006d7b0 msgr2=0x7fbc1006fbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.327+0000 7fbc33a45640 1 --2- 172.21.15.52:0/127587146 >> [v2:172.21.15.111:6804/3267577455,v1:172.21.15.111:6806/3267577455] conn(0x7fbc1006d7b0 0x7fbc1006fbd0 crc :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.327+0000 7fbc33a45640 1 -- 172.21.15.52:0/127587146 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbc100678f0 msgr2=0x7fbc10069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.327+0000 7fbc33a45640 1 --2- 172.21.15.52:0/127587146 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fbc100678f0 0x7fbc10069db0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fbc280af000 tx=0x7fbc28008040 comp rx=0 tx=0).stop 2024-05-04T21:20:37.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.327+0000 7fbc33a45640 1 -- 172.21.15.52:0/127587146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc2c157500 msgr2=0x7fbc2c157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.327+0000 7fbc33a45640 1 --2- 172.21.15.52:0/127587146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc2c157500 0x7fbc2c157980 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c158fd0 tx=0x7fbc2400c960 comp rx=0 tx=0).stop 2024-05-04T21:20:37.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.327+0000 7fbc31fe2640 1 -- 172.21.15.52:0/127587146 reap_dead start 2024-05-04T21:20:37.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.327+0000 7fbc33a45640 1 -- 172.21.15.52:0/127587146 shutdown_connections 2024-05-04T21:20:37.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.327+0000 7fbc33a45640 1 -- 172.21.15.52:0/127587146 >> 172.21.15.52:0/127587146 conn(0x7fbc2c067760 msgr2=0x7fbc2c15c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:37.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.327+0000 7fbc33a45640 1 -- 172.21.15.52:0/127587146 shutdown_connections 2024-05-04T21:20:37.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.327+0000 7fbc33a45640 1 -- 172.21.15.52:0/127587146 wait complete. 2024-05-04T21:20:37.349 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-04T21:20:37.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.467+0000 7f135229b640 1 Processor -- start 2024-05-04T21:20:37.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.467+0000 7f135229b640 1 -- start start 2024-05-04T21:20:37.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f135229b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f134c0aa0c0 0x7f134c0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f135229b640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f134c06d1b0 con 0x7f134c15bf50 2024-05-04T21:20:37.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f135229b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f134c06d490 con 0x7f134c0aa550 2024-05-04T21:20:37.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f135229b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f134c15ca40 con 0x7f134c0aa0c0 2024-05-04T21:20:37.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134bfff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f134c15bf50 0x7f134c15c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51728/0 (socket says 172.21.15.52:51728) 2024-05-04T21:20:37.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134bfff640 1 -- 172.21.15.52:0/4192480373 learned_addr learned my addr 172.21.15.52:0/4192480373 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:37.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134b7fe640 1 --2- 172.21.15.52:0/4192480373 >> v2:172.21.15.111:3300/0 conn(0x7f134c0aa0c0 0x7f134c0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 -- 172.21.15.52:0/4192480373 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 593019033 0 0) 0x7f134c06d1b0 con 0x7f134c15bf50 2024-05-04T21:20:37.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 -- 172.21.15.52:0/4192480373 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1334003530 con 0x7f134c15bf50 2024-05-04T21:20:37.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 -- 172.21.15.52:0/4192480373 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1252992768 0 0) 0x7f1334003530 con 0x7f134c15bf50 2024-05-04T21:20:37.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 -- 172.21.15.52:0/4192480373 >> v2:172.21.15.111:3300/0 conn(0x7f134c0aa0c0 msgr2=0x7f134c0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:37.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 --2- 172.21.15.52:0/4192480373 >> v2:172.21.15.111:3300/0 conn(0x7f134c0aa0c0 0x7f134c0ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 -- 172.21.15.52:0/4192480373 >> v1:172.21.15.111:6789/0 conn(0x7f134c0aa550 legacy=0x7f134c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 -- 172.21.15.52:0/4192480373 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c06d770 con 0x7f134c15bf50 2024-05-04T21:20:37.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 -- 172.21.15.52:0/4192480373 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f133c001a00 con 0x7f134c15bf50 2024-05-04T21:20:37.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 -- 172.21.15.52:0/4192480373 >> v1:172.21.15.52:6789/0 conn(0x7f134c15bf50 legacy=0x7f134c15c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 --2- 172.21.15.52:0/4192480373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13340039c0 0x7f1334003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 --2- 172.21.15.52:0/4192480373 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1334004810 0x7f1334004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 --2- 172.21.15.52:0/4192480373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13340051d0 0x7f133400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 -- 172.21.15.52:0/4192480373 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f134c15ca40 con 0x7f13340051d0 2024-05-04T21:20:37.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 -- 172.21.15.52:0/4192480373 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f133400c070 con 0x7f13340039c0 2024-05-04T21:20:37.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134affd640 1 -- 172.21.15.52:0/4192480373 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f133400c270 con 0x7f1334004810 2024-05-04T21:20:37.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f1350838640 1 --2- 172.21.15.52:0/4192480373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13340051d0 0x7f133400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134bfff640 1 --2- 172.21.15.52:0/4192480373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13340039c0 0x7f1334003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f134b7fe640 1 --2- 172.21.15.52:0/4192480373 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1334004810 0x7f1334004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f1350838640 1 -- 172.21.15.52:0/4192480373 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1334004810 msgr2=0x7f1334004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f1350838640 1 --2- 172.21.15.52:0/4192480373 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1334004810 0x7f1334004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f1350838640 1 -- 172.21.15.52:0/4192480373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13340039c0 msgr2=0x7f1334003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f1350838640 1 --2- 172.21.15.52:0/4192480373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13340039c0 0x7f1334003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f1350838640 1 -- 172.21.15.52:0/4192480373 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1334003530 con 0x7f13340051d0 2024-05-04T21:20:37.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.471+0000 7f1350838640 1 -- 172.21.15.52:0/4192480373 reap_dead start 2024-05-04T21:20:37.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f1350838640 1 --2- 172.21.15.52:0/4192480373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13340051d0 0x7f133400b9d0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f1344001780 tx=0x7f134401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bc3c14112a605c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:37.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f134affd640 1 -- 172.21.15.52:0/4192480373 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13440013a0 con 0x7f13340051d0 2024-05-04T21:20:37.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f134affd640 1 -- 172.21.15.52:0/4192480373 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13440a9070 con 0x7f13340051d0 2024-05-04T21:20:37.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f134affd640 1 -- 172.21.15.52:0/4192480373 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344003600 con 0x7f13340051d0 2024-05-04T21:20:37.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 -- 172.21.15.52:0/4192480373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13340051d0 msgr2=0x7f133400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 --2- 172.21.15.52:0/4192480373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13340051d0 0x7f133400b9d0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f1344001780 tx=0x7f134401c420 comp rx=0 tx=0).stop 2024-05-04T21:20:37.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 -- 172.21.15.52:0/4192480373 shutdown_connections 2024-05-04T21:20:37.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 --2- 172.21.15.52:0/4192480373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13340051d0 0x7f133400b9d0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 -- 172.21.15.52:0/4192480373 >> 172.21.15.52:0/4192480373 conn(0x7f134c0b14a0 msgr2=0x7f134c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:37.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 -- 172.21.15.52:0/4192480373 shutdown_connections 2024-05-04T21:20:37.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 -- 172.21.15.52:0/4192480373 wait complete. 2024-05-04T21:20:37.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 Processor -- start 2024-05-04T21:20:37.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 -- start start 2024-05-04T21:20:37.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13340051d0 0x7f134c151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f134c14f3d0 0x7f134c14f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f134c128560 0x7f134c1289e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f134c06cd60 con 0x7f134c14f3d0 2024-05-04T21:20:37.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f134c06dd30 con 0x7f134c128560 2024-05-04T21:20:37.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f134c06cee0 con 0x7f13340051d0 2024-05-04T21:20:37.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f134b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f134c14f3d0 0x7f134c14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f134b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f134c14f3d0 0x7f134c14f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40740/0 (socket says 172.21.15.52:40740) 2024-05-04T21:20:37.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f134b7fe640 1 -- 172.21.15.52:0/4280829968 learned_addr learned my addr 172.21.15.52:0/4280829968 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:37.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f134bfff640 1 --2- 172.21.15.52:0/4280829968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13340051d0 0x7f134c151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f1350838640 1 --2- 172.21.15.52:0/4280829968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f134c128560 0x7f134c1289e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f134b7fe640 1 -- 172.21.15.52:0/4280829968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13340051d0 msgr2=0x7f134c151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f134b7fe640 1 --2- 172.21.15.52:0/4280829968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13340051d0 0x7f134c151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f134b7fe640 1 -- 172.21.15.52:0/4280829968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f134c128560 msgr2=0x7f134c1289e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f134b7fe640 1 --2- 172.21.15.52:0/4280829968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f134c128560 0x7f134c1289e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f134b7fe640 1 -- 172.21.15.52:0/4280829968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c151d50 con 0x7f134c14f3d0 2024-05-04T21:20:37.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f134b7fe640 1 --2- 172.21.15.52:0/4280829968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f134c14f3d0 0x7f134c14f850 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f13400115c0 tx=0x7f1340011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:37.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f1348ff9640 1 -- 172.21.15.52:0/4280829968 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340015870 con 0x7f134c14f3d0 2024-05-04T21:20:37.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f1348ff9640 1 -- 172.21.15.52:0/4280829968 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1340015a10 con 0x7f134c14f3d0 2024-05-04T21:20:37.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f1348ff9640 1 -- 172.21.15.52:0/4280829968 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340015dc0 con 0x7f134c14f3d0 2024-05-04T21:20:37.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 -- 172.21.15.52:0/4280829968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134c152030 con 0x7f134c14f3d0 2024-05-04T21:20:37.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.475+0000 7f135229b640 1 -- 172.21.15.52:0/4280829968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134c1525f0 con 0x7f134c14f3d0 2024-05-04T21:20:37.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.479+0000 7f135229b640 1 -- 172.21.15.52:0/4280829968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134c062e50 con 0x7f134c14f3d0 2024-05-04T21:20:37.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.479+0000 7f1348ff9640 1 -- 172.21.15.52:0/4280829968 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1340013400 con 0x7f134c14f3d0 2024-05-04T21:20:37.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.479+0000 7f1348ff9640 1 --2- 172.21.15.52:0/4280829968 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13380678c0 0x7f1338069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.479+0000 7f1348ff9640 1 -- 172.21.15.52:0/4280829968 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f1340085e40 con 0x7f134c14f3d0 2024-05-04T21:20:37.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.479+0000 7f134bfff640 1 --2- 172.21.15.52:0/4280829968 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13380678c0 0x7f1338069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.479+0000 7f134bfff640 1 --2- 172.21.15.52:0/4280829968 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13380678c0 0x7f1338069d80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f133c004c30 tx=0x7f133c006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:37.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.483+0000 7f1348ff9640 1 -- 172.21.15.52:0/4280829968 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1340019030 con 0x7f134c14f3d0 2024-05-04T21:20:37.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.639+0000 7f135229b640 1 -- 172.21.15.52:0/4280829968 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f134c075320 con 0x7f13380678c0 2024-05-04T21:20:37.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.639+0000 7f1348ff9640 1 -- 172.21.15.52:0/4280829968 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f134c075320 con 0x7f13380678c0 2024-05-04T21:20:37.644 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 7 to deep-scrub 2024-05-04T21:20:37.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.639+0000 7f135229b640 1 -- 172.21.15.52:0/4280829968 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13380678c0 msgr2=0x7f1338069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.639+0000 7f135229b640 1 --2- 172.21.15.52:0/4280829968 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13380678c0 0x7f1338069d80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f133c004c30 tx=0x7f133c006040 comp rx=0 tx=0).stop 2024-05-04T21:20:37.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.639+0000 7f135229b640 1 -- 172.21.15.52:0/4280829968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f134c14f3d0 msgr2=0x7f134c14f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.639+0000 7f135229b640 1 --2- 172.21.15.52:0/4280829968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f134c14f3d0 0x7f134c14f850 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f13400115c0 tx=0x7f1340011a80 comp rx=0 tx=0).stop 2024-05-04T21:20:37.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.639+0000 7f135229b640 1 -- 172.21.15.52:0/4280829968 shutdown_connections 2024-05-04T21:20:37.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.639+0000 7f135229b640 1 --2- 172.21.15.52:0/4280829968 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f13380678c0 0x7f1338069d80 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.639+0000 7f135229b640 1 --2- 172.21.15.52:0/4280829968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f134c128560 0x7f134c1289e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.639+0000 7f135229b640 1 --2- 172.21.15.52:0/4280829968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f134c14f3d0 0x7f134c14f850 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.639+0000 7f135229b640 1 --2- 172.21.15.52:0/4280829968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13340051d0 0x7f134c151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.639+0000 7f135229b640 1 -- 172.21.15.52:0/4280829968 >> 172.21.15.52:0/4280829968 conn(0x7f134c0b14a0 msgr2=0x7f134c158c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:37.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.639+0000 7f135229b640 1 -- 172.21.15.52:0/4280829968 shutdown_connections 2024-05-04T21:20:37.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.639+0000 7f135229b640 1 -- 172.21.15.52:0/4280829968 wait complete. 2024-05-04T21:20:37.661 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-04T21:20:37.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31b4d9f640 1 Processor -- start 2024-05-04T21:20:37.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31b4d9f640 1 -- start start 2024-05-04T21:20:37.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31b4d9f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f31b01625b0 0x7f31b006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31b4d9f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f31b00ccbc0 0x7f31b00cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31b4d9f640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b0071640 con 0x7f31b00cd990 2024-05-04T21:20:37.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31b4d9f640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f31b00cd570 con 0x7f31b01625b0 2024-05-04T21:20:37.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31b4d9f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f31b0070aa0 con 0x7f31b00ccbc0 2024-05-04T21:20:37.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31ae59c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f31b01625b0 0x7f31b006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31add9b640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f31b00cd990 0x7f31b0070f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51732/0 (socket says 172.21.15.52:51732) 2024-05-04T21:20:37.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31add9b640 1 -- 172.21.15.52:0/2536609773 learned_addr learned my addr 172.21.15.52:0/2536609773 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:37.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31aed9d640 1 --2- 172.21.15.52:0/2536609773 >> v2:172.21.15.111:3300/0 conn(0x7f31b00ccbc0 0x7f31b00cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31ae59c640 1 -- 172.21.15.52:0/2536609773 >> v2:172.21.15.111:3300/0 conn(0x7f31b00ccbc0 msgr2=0x7f31b00cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:37.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31ae59c640 1 --2- 172.21.15.52:0/2536609773 >> v2:172.21.15.111:3300/0 conn(0x7f31b00ccbc0 0x7f31b00cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31ae59c640 1 -- 172.21.15.52:0/2536609773 >> v1:172.21.15.52:6789/0 conn(0x7f31b00cd990 legacy=0x7f31b0070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.771+0000 7f31ae59c640 1 -- 172.21.15.52:0/2536609773 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b0166ef0 con 0x7f31b01625b0 2024-05-04T21:20:37.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31ae59c640 1 --2- 172.21.15.52:0/2536609773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31b01625b0 0x7f31b006eab0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f31b006b5a0 tx=0x7f31a001c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecc3d16aa4e37f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:37.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31ad59a640 1 -- 172.21.15.52:0/2536609773 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a00b0070 con 0x7f31b01625b0 2024-05-04T21:20:37.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31ad59a640 1 -- 172.21.15.52:0/2536609773 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a00ab600 con 0x7f31b01625b0 2024-05-04T21:20:37.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31ad59a640 1 -- 172.21.15.52:0/2536609773 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a00ab9d0 con 0x7f31b01625b0 2024-05-04T21:20:37.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 -- 172.21.15.52:0/2536609773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31b01625b0 msgr2=0x7f31b006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 --2- 172.21.15.52:0/2536609773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31b01625b0 0x7f31b006eab0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f31b006b5a0 tx=0x7f31a001c310 comp rx=0 tx=0).stop 2024-05-04T21:20:37.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 -- 172.21.15.52:0/2536609773 shutdown_connections 2024-05-04T21:20:37.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 --2- 172.21.15.52:0/2536609773 >> v2:172.21.15.111:3300/0 conn(0x7f31b00ccbc0 0x7f31b00cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 --2- 172.21.15.52:0/2536609773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31b01625b0 0x7f31b006eab0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 -- 172.21.15.52:0/2536609773 >> 172.21.15.52:0/2536609773 conn(0x7f31b00b14a0 msgr2=0x7f31b00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:37.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 -- 172.21.15.52:0/2536609773 shutdown_connections 2024-05-04T21:20:37.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 -- 172.21.15.52:0/2536609773 wait complete. 2024-05-04T21:20:37.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 Processor -- start 2024-05-04T21:20:37.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 -- start start 2024-05-04T21:20:37.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31b00ccbc0 0x7f31b01538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31b00cd990 0x7f31b0154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31b01625b0 0x7f31b0155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f31b006e600 con 0x7f31b00cd990 2024-05-04T21:20:37.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01671d0 con 0x7f31b00ccbc0 2024-05-04T21:20:37.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f31b0162990 con 0x7f31b01625b0 2024-05-04T21:20:37.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31add9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31b00cd990 0x7f31b0154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31add9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31b00cd990 0x7f31b0154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40768/0 (socket says 172.21.15.52:40768) 2024-05-04T21:20:37.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31add9b640 1 -- 172.21.15.52:0/3734922887 learned_addr learned my addr 172.21.15.52:0/3734922887 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:37.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31ae59c640 1 --2- 172.21.15.52:0/3734922887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31b00ccbc0 0x7f31b01538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31aed9d640 1 --2- 172.21.15.52:0/3734922887 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31b01625b0 0x7f31b0155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31add9b640 1 -- 172.21.15.52:0/3734922887 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31b01625b0 msgr2=0x7f31b0155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31add9b640 1 --2- 172.21.15.52:0/3734922887 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31b01625b0 0x7f31b0155530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31add9b640 1 -- 172.21.15.52:0/3734922887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31b00ccbc0 msgr2=0x7f31b01538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31add9b640 1 --2- 172.21.15.52:0/3734922887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31b00ccbc0 0x7f31b01538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31add9b640 1 -- 172.21.15.52:0/3734922887 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b01543a0 con 0x7f31b00cd990 2024-05-04T21:20:37.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31add9b640 1 --2- 172.21.15.52:0/3734922887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31b00cd990 0x7f31b0154ff0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f319c00ee30 tx=0x7f319c00c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:37.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31a6ffd640 1 -- 172.21.15.52:0/3734922887 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c019070 con 0x7f31b00cd990 2024-05-04T21:20:37.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31a6ffd640 1 -- 172.21.15.52:0/3734922887 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f319c007af0 con 0x7f31b00cd990 2024-05-04T21:20:37.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 -- 172.21.15.52:0/3734922887 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b01f9620 con 0x7f31b00cd990 2024-05-04T21:20:37.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31a6ffd640 1 -- 172.21.15.52:0/3734922887 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c00ce10 con 0x7f31b00cd990 2024-05-04T21:20:37.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.775+0000 7f31b4d9f640 1 -- 172.21.15.52:0/3734922887 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b01f9be0 con 0x7f31b00cd990 2024-05-04T21:20:37.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.779+0000 7f31b4d9f640 1 -- 172.21.15.52:0/3734922887 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b0062e50 con 0x7f31b00cd990 2024-05-04T21:20:37.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.779+0000 7f31a6ffd640 1 -- 172.21.15.52:0/3734922887 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f319c01c070 con 0x7f31b00cd990 2024-05-04T21:20:37.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.779+0000 7f31a6ffd640 1 --2- 172.21.15.52:0/3734922887 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f31980678c0 0x7f3198069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:37.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.779+0000 7f31a6ffd640 1 -- 172.21.15.52:0/3734922887 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f319c0875f0 con 0x7f31b00cd990 2024-05-04T21:20:37.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.779+0000 7f31ae59c640 1 --2- 172.21.15.52:0/3734922887 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f31980678c0 0x7f3198069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:37.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.779+0000 7f31ae59c640 1 --2- 172.21.15.52:0/3734922887 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f31980678c0 0x7f3198069d80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f31a00ad000 tx=0x7f31a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:37.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.783+0000 7f31a6ffd640 1 -- 172.21.15.52:0/3734922887 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f319c057df0 con 0x7f31b00cd990 2024-05-04T21:20:37.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.939+0000 7f31b4d9f640 1 -- 172.21.15.52:0/3734922887 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31b0155b90 con 0x7f31980678c0 2024-05-04T21:20:37.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.939+0000 7f31a6ffd640 1 -- 172.21.15.52:0/3734922887 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13505 (secure 0 0 0) 0x7f31b0155b90 con 0x7f31980678c0 2024-05-04T21:20:37.945 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:37.945 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-04T21:20:37.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.943+0000 7f31b4d9f640 1 -- 172.21.15.52:0/3734922887 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f31980678c0 msgr2=0x7f3198069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.943+0000 7f31b4d9f640 1 --2- 172.21.15.52:0/3734922887 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f31980678c0 0x7f3198069d80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f31a00ad000 tx=0x7f31a0008040 comp rx=0 tx=0).stop 2024-05-04T21:20:37.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.943+0000 7f31b4d9f640 1 -- 172.21.15.52:0/3734922887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31b00cd990 msgr2=0x7f31b0154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:37.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.943+0000 7f31b4d9f640 1 --2- 172.21.15.52:0/3734922887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31b00cd990 0x7f31b0154ff0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f319c00ee30 tx=0x7f319c00c430 comp rx=0 tx=0).stop 2024-05-04T21:20:37.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.943+0000 7f31b4d9f640 1 -- 172.21.15.52:0/3734922887 shutdown_connections 2024-05-04T21:20:37.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.943+0000 7f31b4d9f640 1 --2- 172.21.15.52:0/3734922887 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f31980678c0 0x7f3198069d80 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.943+0000 7f31b4d9f640 1 --2- 172.21.15.52:0/3734922887 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31b01625b0 0x7f31b0155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.943+0000 7f31b4d9f640 1 --2- 172.21.15.52:0/3734922887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31b00cd990 0x7f31b0154ff0 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.943+0000 7f31b4d9f640 1 --2- 172.21.15.52:0/3734922887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31b00ccbc0 0x7f31b01538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:37.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.943+0000 7f31b4d9f640 1 -- 172.21.15.52:0/3734922887 >> 172.21.15.52:0/3734922887 conn(0x7f31b00b14a0 msgr2=0x7f31b0164630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:37.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.943+0000 7f31b4d9f640 1 -- 172.21.15.52:0/3734922887 shutdown_connections 2024-05-04T21:20:37.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:37.943+0000 7f31b4d9f640 1 -- 172.21.15.52:0/3734922887 wait complete. 2024-05-04T21:20:37.962 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":16298,"stamp":"2024-05-04T21:20:36.862968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":11699948,"kb_used_data":4520,"kb_used_omap":4746,"kb_used_meta":11690613,"kb_avail":743274772,"statfs":{"total":773094113280,"available":761113366528,"internally_reserved":0,"allocated":4628480,"data_stored":2533520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4860773,"internal_metadata":11971187867},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":376,"apply_latency_ms":376,"commit_latency_ns":376000000,"apply_latency_ns":376000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.656811"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":469,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-04T21:20:26.282615+0000","last_change":"2024-05-04T12:18:12.644577+0000","last_active":"2024-05-04T21:20:26.282615+0000","last_peered":"2024-05-04T21:20:26.282615+0000","last_clean":"2024-05-04T21:20:26.282615+0000","last_became_active":"2024-05-04T12:18:12.640041+0000","last_became_peered":"2024-05-04T12:18:12.640041+0000","last_unstale":"2024-05-04T21:20:26.282615+0000","last_undegraded":"2024-05-04T21:20:26.282615+0000","last_fullsized":"2024-05-04T21:20:26.282615+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T12:18:11.260157+0000","last_clean_scrub_stamp":"2024-05-04T12:18:11.260157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T00:15:02.089387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834581361,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1462452,"kb_used_data":1000,"kb_used_omap":126,"kb_used_meta":1461313,"kb_avail":92909388,"statfs":{"total":96636764160,"available":95139213312,"internally_reserved":0,"allocated":1024000,"data_stored":759466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":129091,"internal_metadata":1496385469},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834581361,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1637420,"kb_used_data":420,"kb_used_omap":796,"kb_used_meta":1636195,"kb_avail":92734420,"statfs":{"total":96636764160,"available":94960046080,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":815583,"internal_metadata":1675464225},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834581360,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1498156,"kb_used_data":420,"kb_used_omap":583,"kb_used_meta":1497144,"kb_avail":92873684,"statfs":{"total":96636764160,"available":95102652416,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":597575,"internal_metadata":1533075897},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834581360,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1462188,"kb_used_data":420,"kb_used_omap":836,"kb_used_meta":1460923,"kb_avail":92909652,"statfs":{"total":96636764160,"available":95139483648,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":856296,"internal_metadata":1495985944},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834581348,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1608236,"kb_used_data":420,"kb_used_omap":888,"kb_used_meta":1606919,"kb_avail":92763604,"statfs":{"total":96636764160,"available":94989930496,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":909648,"internal_metadata":1645485744},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834581353,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1251052,"kb_used_data":420,"kb_used_omap":363,"kb_used_meta":1250260,"kb_avail":93120788,"statfs":{"total":96636764160,"available":95355686912,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":372172,"internal_metadata":1280266804},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834581350,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1340268,"kb_used_data":420,"kb_used_omap":926,"kb_used_meta":1338913,"kb_avail":93031572,"statfs":{"total":96636764160,"available":95264329728,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":948664,"internal_metadata":1371047496},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":153,"apply_latency_ms":153,"commit_latency_ns":153000000,"apply_latency_ns":153000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834581356,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1440176,"kb_used_data":1000,"kb_used_omap":226,"kb_used_meta":1438941,"kb_avail":92931664,"statfs":{"total":96636764160,"available":95162023936,"internally_reserved":0,"allocated":1024000,"data_stored":759466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":231744,"internal_metadata":1473476288},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":223,"apply_latency_ms":223,"commit_latency_ns":223000000,"apply_latency_ns":223000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T21:20:37.965 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-04T12:18:11.260157+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=4, tm_hour=12, tm_min=18, tm_sec=11, tm_wday=5, tm_yday=125, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=4, tm_hour=21, tm_min=20, tm_sec=32, tm_wday=5, tm_yday=125, tm_isdst=0) 2024-05-04T21:20:37.965 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-04T21:20:57.966 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-04T21:20:58.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f46721640 1 Processor -- start 2024-05-04T21:20:58.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f46721640 1 -- start start 2024-05-04T21:20:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f46721640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4f400aa0c0 0x7f4f400ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f46721640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4f400aa550 0x7f4f400cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f46721640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f4006d1b0 con 0x7f4f4015bf50 2024-05-04T21:20:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f46721640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f4f4015ca40 con 0x7f4f400aa0c0 2024-05-04T21:20:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f46721640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4006d490 con 0x7f4f400aa550 2024-05-04T21:20:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f3f7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4f400aa0c0 0x7f4f400ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f3f7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4f400aa0c0 0x7f4f400ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43724/0 (socket says 172.21.15.52:43724) 2024-05-04T21:20:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f3f7fe640 1 -- 172.21.15.52:0/2841678095 learned_addr learned my addr 172.21.15.52:0/2841678095 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f44cbe640 1 --2- 172.21.15.52:0/2841678095 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4f400aa550 0x7f4f400cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f3f7fe640 1 -- 172.21.15.52:0/2841678095 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4f400aa550 msgr2=0x7f4f400cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f3f7fe640 1 --2- 172.21.15.52:0/2841678095 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4f400aa550 0x7f4f400cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f3f7fe640 1 -- 172.21.15.52:0/2841678095 >> v1:172.21.15.52:6789/0 conn(0x7f4f4015bf50 legacy=0x7f4f4015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f3effd640 1 -- 172.21.15.52:0/2841678095 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1308617364 0 0) 0x7f4f4006d1b0 con 0x7f4f4015bf50 2024-05-04T21:20:58.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f3f7fe640 1 -- 172.21.15.52:0/2841678095 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4006d770 con 0x7f4f400aa0c0 2024-05-04T21:20:58.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.099+0000 7f4f44cbe640 1 --2- 172.21.15.52:0/2841678095 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4f400aa550 0x7f4f400cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:58.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f3f7fe640 1 --2- 172.21.15.52:0/2841678095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f400aa0c0 0x7f4f400ccb50 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009870 tx=0x7f4f3401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7510d6f17510c06 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:58.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f3effd640 1 -- 172.21.15.52:0/2841678095 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f340a9610 con 0x7f4f400aa0c0 2024-05-04T21:20:58.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f3effd640 1 -- 172.21.15.52:0/2841678095 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f340a97b0 con 0x7f4f400aa0c0 2024-05-04T21:20:58.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f3effd640 1 -- 172.21.15.52:0/2841678095 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f340a9b80 con 0x7f4f400aa0c0 2024-05-04T21:20:58.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 -- 172.21.15.52:0/2841678095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f400aa0c0 msgr2=0x7f4f400ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 --2- 172.21.15.52:0/2841678095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f400aa0c0 0x7f4f400ccb50 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009870 tx=0x7f4f3401c530 comp rx=0 tx=0).stop 2024-05-04T21:20:58.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 -- 172.21.15.52:0/2841678095 shutdown_connections 2024-05-04T21:20:58.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 --2- 172.21.15.52:0/2841678095 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4f400aa550 0x7f4f400cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 --2- 172.21.15.52:0/2841678095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f400aa0c0 0x7f4f400ccb50 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 -- 172.21.15.52:0/2841678095 >> 172.21.15.52:0/2841678095 conn(0x7f4f400b14a0 msgr2=0x7f4f400b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:58.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 -- 172.21.15.52:0/2841678095 shutdown_connections 2024-05-04T21:20:58.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 -- 172.21.15.52:0/2841678095 wait complete. 2024-05-04T21:20:58.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 Processor -- start 2024-05-04T21:20:58.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 -- start start 2024-05-04T21:20:58.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4f400aa0c0 0x7f4f400b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f400aa550 0x7f4f401486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f4015bf50 0x7f4f40148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4006ebe0 con 0x7f4f4015bf50 2024-05-04T21:20:58.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4006ea60 con 0x7f4f400aa550 2024-05-04T21:20:58.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f46721640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4006ed60 con 0x7f4f400aa0c0 2024-05-04T21:20:58.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f44cbe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f4015bf50 0x7f4f40148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f44cbe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f4015bf50 0x7f4f40148bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43738/0 (socket says 172.21.15.52:43738) 2024-05-04T21:20:58.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f44cbe640 1 -- 172.21.15.52:0/3628225280 learned_addr learned my addr 172.21.15.52:0/3628225280 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:58.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f3ffff640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4f400aa0c0 0x7f4f400b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f3f7fe640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f400aa550 0x7f4f401486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f44cbe640 1 -- 172.21.15.52:0/3628225280 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4f400aa0c0 msgr2=0x7f4f400b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f44cbe640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4f400aa0c0 0x7f4f400b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f44cbe640 1 -- 172.21.15.52:0/3628225280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f400aa550 msgr2=0x7f4f401486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f44cbe640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f400aa550 0x7f4f401486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f44cbe640 1 -- 172.21.15.52:0/3628225280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f40071130 con 0x7f4f4015bf50 2024-05-04T21:20:58.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f3f7fe640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f400aa550 0x7f4f401486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:58.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.103+0000 7f4f44cbe640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f4015bf50 0x7f4f40148bf0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800cf20 tx=0x7f4f3800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:58.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.107+0000 7f4f3cff9640 1 -- 172.21.15.52:0/3628225280 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3801a070 con 0x7f4f4015bf50 2024-05-04T21:20:58.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.107+0000 7f4f3cff9640 1 -- 172.21.15.52:0/3628225280 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f3800d840 con 0x7f4f4015bf50 2024-05-04T21:20:58.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.107+0000 7f4f3cff9640 1 -- 172.21.15.52:0/3628225280 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3800dc30 con 0x7f4f4015bf50 2024-05-04T21:20:58.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.107+0000 7f4f46721640 1 -- 172.21.15.52:0/3628225280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f40071410 con 0x7f4f4015bf50 2024-05-04T21:20:58.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.107+0000 7f4f46721640 1 -- 172.21.15.52:0/3628225280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f400719d0 con 0x7f4f4015bf50 2024-05-04T21:20:58.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.107+0000 7f4f46721640 1 -- 172.21.15.52:0/3628225280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f40062e50 con 0x7f4f4015bf50 2024-05-04T21:20:58.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.107+0000 7f4f3cff9640 1 -- 172.21.15.52:0/3628225280 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4f3801d070 con 0x7f4f4015bf50 2024-05-04T21:20:58.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.107+0000 7f4f3cff9640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4f2c0678c0 0x7f4f2c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.107+0000 7f4f3cff9640 1 -- 172.21.15.52:0/3628225280 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f4f38086240 con 0x7f4f4015bf50 2024-05-04T21:20:58.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.107+0000 7f4f3ffff640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4f2c0678c0 0x7f4f2c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.111+0000 7f4f3cff9640 1 -- 172.21.15.52:0/3628225280 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4f38057080 con 0x7f4f4015bf50 2024-05-04T21:20:58.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.115+0000 7f4f3ffff640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4f2c0678c0 0x7f4f2c069d80 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4f30000f70 tx=0x7f4f30009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:58.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.271+0000 7f4f46721640 1 -- 172.21.15.52:0/3628225280 --> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4f40075320 con 0x7f4f2c0678c0 2024-05-04T21:20:58.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.271+0000 7f4f3cff9640 1 -- 172.21.15.52:0/3628225280 <== mgr.4101 v2:172.21.15.52:6832/405834661 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13453 (secure 0 0 0) 0x7f4f40075320 con 0x7f4f2c0678c0 2024-05-04T21:20:58.278 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:58.278 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-04T21:20:58.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.275+0000 7f4f46721640 1 -- 172.21.15.52:0/3628225280 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4f2c0678c0 msgr2=0x7f4f2c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.275+0000 7f4f46721640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4f2c0678c0 0x7f4f2c069d80 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4f30000f70 tx=0x7f4f30009040 comp rx=0 tx=0).stop 2024-05-04T21:20:58.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.275+0000 7f4f46721640 1 -- 172.21.15.52:0/3628225280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f4015bf50 msgr2=0x7f4f40148bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.275+0000 7f4f46721640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f4015bf50 0x7f4f40148bf0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800cf20 tx=0x7f4f3800a7d0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.275+0000 7f4f46721640 1 -- 172.21.15.52:0/3628225280 shutdown_connections 2024-05-04T21:20:58.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.275+0000 7f4f46721640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f4f2c0678c0 0x7f4f2c069d80 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.275+0000 7f4f46721640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f4015bf50 0x7f4f40148bf0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.275+0000 7f4f46721640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f400aa550 0x7f4f401486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.275+0000 7f4f46721640 1 --2- 172.21.15.52:0/3628225280 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4f400aa0c0 0x7f4f400b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.275+0000 7f4f46721640 1 -- 172.21.15.52:0/3628225280 >> 172.21.15.52:0/3628225280 conn(0x7f4f400b14a0 msgr2=0x7f4f400cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:58.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.275+0000 7f4f46721640 1 -- 172.21.15.52:0/3628225280 shutdown_connections 2024-05-04T21:20:58.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.275+0000 7f4f46721640 1 -- 172.21.15.52:0/3628225280 wait complete. 2024-05-04T21:20:58.297 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":16308,"stamp":"2024-05-04T21:20:56.868128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":11699944,"kb_used_data":4520,"kb_used_omap":4746,"kb_used_meta":11690613,"kb_avail":743274776,"statfs":{"total":773094113280,"available":761113370624,"internally_reserved":0,"allocated":4628480,"data_stored":2533520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4860773,"internal_metadata":11971187867},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003130"},"pg_stats":[{"pgid":"1.0","version":"37'119","reported_seq":481,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-04T21:20:38.551545+0000","last_change":"2024-05-04T21:20:38.551545+0000","last_active":"2024-05-04T21:20:38.551545+0000","last_peered":"2024-05-04T21:20:38.551545+0000","last_clean":"2024-05-04T21:20:38.551545+0000","last_became_active":"2024-05-04T12:18:12.640041+0000","last_became_peered":"2024-05-04T12:18:12.640041+0000","last_unstale":"2024-05-04T21:20:38.551545+0000","last_undegraded":"2024-05-04T21:20:38.551545+0000","last_fullsized":"2024-05-04T21:20:38.551545+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'119","last_scrub_stamp":"2024-05-04T21:20:38.551189+0000","last_deep_scrub":"37'119","last_deep_scrub_stamp":"2024-05-04T21:20:38.551189+0000","last_clean_scrub_stamp":"2024-05-04T21:20:38.551189+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-06T08:43:02.289577+0000","scrub_duration":375,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834581365,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1462448,"kb_used_data":1000,"kb_used_omap":126,"kb_used_meta":1461313,"kb_avail":92909392,"statfs":{"total":96636764160,"available":95139217408,"internally_reserved":0,"allocated":1024000,"data_stored":759466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":129091,"internal_metadata":1496385469},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834581365,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1637420,"kb_used_data":420,"kb_used_omap":796,"kb_used_meta":1636195,"kb_avail":92734420,"statfs":{"total":96636764160,"available":94960046080,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":815583,"internal_metadata":1675464225},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834581364,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1498156,"kb_used_data":420,"kb_used_omap":583,"kb_used_meta":1497144,"kb_avail":92873684,"statfs":{"total":96636764160,"available":95102652416,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":597575,"internal_metadata":1533075897},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834581364,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1462188,"kb_used_data":420,"kb_used_omap":836,"kb_used_meta":1460923,"kb_avail":92909652,"statfs":{"total":96636764160,"available":95139483648,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":856296,"internal_metadata":1495985944},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834581352,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1608236,"kb_used_data":420,"kb_used_omap":888,"kb_used_meta":1606919,"kb_avail":92763604,"statfs":{"total":96636764160,"available":94989930496,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":909648,"internal_metadata":1645485744},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834581357,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1251052,"kb_used_data":420,"kb_used_omap":363,"kb_used_meta":1250260,"kb_avail":93120788,"statfs":{"total":96636764160,"available":95355686912,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":372172,"internal_metadata":1280266804},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834581354,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1340268,"kb_used_data":420,"kb_used_omap":926,"kb_used_meta":1338913,"kb_avail":93031572,"statfs":{"total":96636764160,"available":95264329728,"internally_reserved":0,"allocated":430080,"data_stored":169098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":948664,"internal_metadata":1371047496},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834581360,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1440176,"kb_used_data":1000,"kb_used_omap":226,"kb_used_meta":1438941,"kb_avail":92931664,"statfs":{"total":96636764160,"available":95162023936,"internally_reserved":0,"allocated":1024000,"data_stored":759466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":231744,"internal_metadata":1473476288},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T21:20:58.298 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-04T21:20:58.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5bdc70640 1 Processor -- start 2024-05-04T21:20:58.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5bdc70640 1 -- start start 2024-05-04T21:20:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5bdc70640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5b815c590 0x7fc5b80cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5bdc70640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc5b80cccd0 0x7fc5b806e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5bdc70640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5b806dbf0 con 0x7fc5b8145960 2024-05-04T21:20:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5bdc70640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc5b806cfe0 con 0x7fc5b80cccd0 2024-05-04T21:20:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5bdc70640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b806d2c0 con 0x7fc5b815c590 2024-05-04T21:20:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5b7fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc5b80cccd0 0x7fc5b806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5b77fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc5b8145960 0x7fc5b8160590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55894/0 (socket says 172.21.15.52:55894) 2024-05-04T21:20:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5b77fe640 1 -- 172.21.15.52:0/4061697978 learned_addr learned my addr 172.21.15.52:0/4061697978 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5b7fff640 1 --2- 172.21.15.52:0/4061697978 >> v2:172.21.15.52:3300/0 conn(0x7fc5b80cccd0 0x7fc5b806e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43742/0 (socket says 172.21.15.52:43742) 2024-05-04T21:20:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5b6ffd640 1 --2- 172.21.15.52:0/4061697978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5b815c590 0x7fc5b80cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5b7fff640 1 -- 172.21.15.52:0/4061697978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5b815c590 msgr2=0x7fc5b80cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5b7fff640 1 --2- 172.21.15.52:0/4061697978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5b815c590 0x7fc5b80cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5b7fff640 1 -- 172.21.15.52:0/4061697978 >> v1:172.21.15.52:6789/0 conn(0x7fc5b8145960 legacy=0x7fc5b8160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.419+0000 7fc5b7fff640 1 -- 172.21.15.52:0/4061697978 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b806d5a0 con 0x7fc5b80cccd0 2024-05-04T21:20:58.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.423+0000 7fc5b7fff640 1 --2- 172.21.15.52:0/4061697978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5b80cccd0 0x7fc5b806e080 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80097c0 tx=0x7fc5a801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fd6df7e0891fe0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:58.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.423+0000 7fc5b67fc640 1 -- 172.21.15.52:0/4061697978 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a80b0070 con 0x7fc5b80cccd0 2024-05-04T21:20:58.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.423+0000 7fc5b67fc640 1 -- 172.21.15.52:0/4061697978 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a80ab600 con 0x7fc5b80cccd0 2024-05-04T21:20:58.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.423+0000 7fc5b67fc640 1 -- 172.21.15.52:0/4061697978 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a80ab9d0 con 0x7fc5b80cccd0 2024-05-04T21:20:58.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.423+0000 7fc5bdc70640 1 -- 172.21.15.52:0/4061697978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5b80cccd0 msgr2=0x7fc5b806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.423+0000 7fc5bdc70640 1 --2- 172.21.15.52:0/4061697978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5b80cccd0 0x7fc5b806e080 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80097c0 tx=0x7fc5a801c410 comp rx=0 tx=0).stop 2024-05-04T21:20:58.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.423+0000 7fc5bdc70640 1 -- 172.21.15.52:0/4061697978 shutdown_connections 2024-05-04T21:20:58.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.423+0000 7fc5bdc70640 1 --2- 172.21.15.52:0/4061697978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5b80cccd0 0x7fc5b806e080 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.423+0000 7fc5bdc70640 1 --2- 172.21.15.52:0/4061697978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5b815c590 0x7fc5b80cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.423+0000 7fc5bdc70640 1 -- 172.21.15.52:0/4061697978 >> 172.21.15.52:0/4061697978 conn(0x7fc5b8067730 msgr2=0x7fc5b80b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.423+0000 7fc5bdc70640 1 -- 172.21.15.52:0/4061697978 shutdown_connections 2024-05-04T21:20:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.423+0000 7fc5bdc70640 1 -- 172.21.15.52:0/4061697978 wait complete. 2024-05-04T21:20:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.423+0000 7fc5bdc70640 1 Processor -- start 2024-05-04T21:20:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5bdc70640 1 -- start start 2024-05-04T21:20:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5bdc70640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5b80cccd0 0x7fc5b80a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5bdc70640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5b8145960 0x7fc5b80aa290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5bdc70640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5b815c590 0x7fc5b80afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5bdc70640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b8145ec0 con 0x7fc5b8145960 2024-05-04T21:20:58.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5bdc70640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b8145d40 con 0x7fc5b815c590 2024-05-04T21:20:58.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5bdc70640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b806edd0 con 0x7fc5b80cccd0 2024-05-04T21:20:58.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5b6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5b8145960 0x7fc5b80aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5b6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5b8145960 0x7fc5b80aa290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43750/0 (socket says 172.21.15.52:43750) 2024-05-04T21:20:58.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5b6ffd640 1 -- 172.21.15.52:0/2129284047 learned_addr learned my addr 172.21.15.52:0/2129284047 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:58.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5b77fe640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5b80cccd0 0x7fc5b80a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5b7fff640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5b815c590 0x7fc5b80afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5b6ffd640 1 -- 172.21.15.52:0/2129284047 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5b80cccd0 msgr2=0x7fc5b80a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5b6ffd640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5b80cccd0 0x7fc5b80a9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5b6ffd640 1 -- 172.21.15.52:0/2129284047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5b815c590 msgr2=0x7fc5b80afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5b6ffd640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5b815c590 0x7fc5b80afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5b6ffd640 1 -- 172.21.15.52:0/2129284047 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b80aefc0 con 0x7fc5b8145960 2024-05-04T21:20:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5b7fff640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5b815c590 0x7fc5b80afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5b6ffd640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5b8145960 0x7fc5b80aa290 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00d800 tx=0x7fc5ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc59ffff640 1 -- 172.21.15.52:0/2129284047 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00b6c0 con 0x7fc5b8145960 2024-05-04T21:20:58.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc59ffff640 1 -- 172.21.15.52:0/2129284047 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5ac019070 con 0x7fc5b8145960 2024-05-04T21:20:58.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5bdc70640 1 -- 172.21.15.52:0/2129284047 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b80aea00 con 0x7fc5b8145960 2024-05-04T21:20:58.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc59ffff640 1 -- 172.21.15.52:0/2129284047 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00ba80 con 0x7fc5b8145960 2024-05-04T21:20:58.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5bdc70640 1 -- 172.21.15.52:0/2129284047 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b80ada10 con 0x7fc5b8145960 2024-05-04T21:20:58.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc5bdc70640 1 -- 172.21.15.52:0/2129284047 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b8062e50 con 0x7fc5b8145960 2024-05-04T21:20:58.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc59ffff640 1 -- 172.21.15.52:0/2129284047 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc5ac00bc20 con 0x7fc5b8145960 2024-05-04T21:20:58.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.427+0000 7fc59ffff640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5a40678c0 0x7fc5a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.431+0000 7fc59ffff640 1 -- 172.21.15.52:0/2129284047 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fc5ac085d20 con 0x7fc5b8145960 2024-05-04T21:20:58.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.431+0000 7fc5b77fe640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5a40678c0 0x7fc5a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.431+0000 7fc5b77fe640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5a40678c0 0x7fc5a4069d80 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc5b00012f0 tx=0x7fc5b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:58.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.435+0000 7fc59ffff640 1 -- 172.21.15.52:0/2129284047 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc5ac056bf0 con 0x7fc5b8145960 2024-05-04T21:20:58.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.587+0000 7fc5bdc70640 1 -- 172.21.15.52:0/2129284047 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fc5b8075320 con 0x7fc5b8145960 2024-05-04T21:20:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.595+0000 7fc59ffff640 1 -- 172.21.15.52:0/2129284047 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7fc5ac0564f0 con 0x7fc5b8145960 2024-05-04T21:20:58.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.595+0000 7fc59ffff640 1 -- 172.21.15.52:0/2129284047 <== mon.0 v2:172.21.15.52:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc5ac089030 con 0x7fc5b8145960 2024-05-04T21:20:58.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.595+0000 7fc5bdc70640 1 -- 172.21.15.52:0/2129284047 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5a40678c0 msgr2=0x7fc5a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.595+0000 7fc5bdc70640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5a40678c0 0x7fc5a4069d80 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc5b00012f0 tx=0x7fc5b0009040 comp rx=0 tx=0).stop 2024-05-04T21:20:58.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.595+0000 7fc5bdc70640 1 -- 172.21.15.52:0/2129284047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5b8145960 msgr2=0x7fc5b80aa290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.595+0000 7fc5bdc70640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5b8145960 0x7fc5b80aa290 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00d800 tx=0x7fc5ac00dcc0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.595+0000 7fc5bdc70640 1 -- 172.21.15.52:0/2129284047 shutdown_connections 2024-05-04T21:20:58.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.595+0000 7fc5bdc70640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7fc5a40678c0 0x7fc5a4069d80 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.595+0000 7fc5bdc70640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5b815c590 0x7fc5b80afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.595+0000 7fc5bdc70640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5b8145960 0x7fc5b80aa290 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.595+0000 7fc5bdc70640 1 --2- 172.21.15.52:0/2129284047 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc5b80cccd0 0x7fc5b80a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.595+0000 7fc5bdc70640 1 -- 172.21.15.52:0/2129284047 >> 172.21.15.52:0/2129284047 conn(0x7fc5b8067730 msgr2=0x7fc5b815d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:58.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.595+0000 7fc5bdc70640 1 -- 172.21.15.52:0/2129284047 shutdown_connections 2024-05-04T21:20:58.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.595+0000 7fc5bdc70640 1 -- 172.21.15.52:0/2129284047 wait complete. 2024-05-04T21:20:58.617 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-04T21:20:58.618 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-04T21:20:58.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f725ccf2640 1 Processor -- start 2024-05-04T21:20:58.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f725ccf2640 1 -- start start 2024-05-04T21:20:58.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f725ccf2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f725815bf50 0x7f725815c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f725ccf2640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72580aa0c0 0x7f72580aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f725ccf2640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f725806d1b0 con 0x7f72580ccb50 2024-05-04T21:20:58.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f725ccf2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f725806d490 con 0x7f725815bf50 2024-05-04T21:20:58.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f725ccf2640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f725806d770 con 0x7f72580aa0c0 2024-05-04T21:20:58.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7256d9d640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f72580ccb50 0x7f725806e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55902/0 (socket says 172.21.15.52:55902) 2024-05-04T21:20:58.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7256d9d640 1 -- 172.21.15.52:0/1848169177 learned_addr learned my addr 172.21.15.52:0/1848169177 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-04T21:20:58.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f724ffff640 1 --2- 172.21.15.52:0/1848169177 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72580aa0c0 0x7f72580aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f725659c640 1 --2- 172.21.15.52:0/1848169177 >> v2:172.21.15.111:3300/0 conn(0x7f725815bf50 0x7f725815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 -- 172.21.15.52:0/1848169177 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2584084314 0 0) 0x7f725806d1b0 con 0x7f72580ccb50 2024-05-04T21:20:58.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 -- 172.21.15.52:0/1848169177 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7244003530 con 0x7f72580ccb50 2024-05-04T21:20:58.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 -- 172.21.15.52:0/1848169177 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1033283216 0 0) 0x7f7244003530 con 0x7f72580ccb50 2024-05-04T21:20:58.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 -- 172.21.15.52:0/1848169177 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72580aa0c0 msgr2=0x7f72580aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:58.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 --2- 172.21.15.52:0/1848169177 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72580aa0c0 0x7f72580aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 -- 172.21.15.52:0/1848169177 >> v2:172.21.15.111:3300/0 conn(0x7f725815bf50 msgr2=0x7f725815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:58.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 --2- 172.21.15.52:0/1848169177 >> v2:172.21.15.111:3300/0 conn(0x7f725815bf50 0x7f725815c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 -- 172.21.15.52:0/1848169177 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725806da50 con 0x7f72580ccb50 2024-05-04T21:20:58.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 -- 172.21.15.52:0/1848169177 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3255576533 0 0) 0x7f7250001a00 con 0x7f72580ccb50 2024-05-04T21:20:58.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 -- 172.21.15.52:0/1848169177 >> v1:172.21.15.52:6789/0 conn(0x7f72580ccb50 legacy=0x7f725806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 --2- 172.21.15.52:0/1848169177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72440039c0 0x7f7244003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 --2- 172.21.15.52:0/1848169177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7244004810 0x7f7244004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 --2- 172.21.15.52:0/1848169177 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72580aa0c0 0x7f72440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 -- 172.21.15.52:0/1848169177 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f725806d490 con 0x7f7244004810 2024-05-04T21:20:58.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 -- 172.21.15.52:0/1848169177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f725806d770 con 0x7f72440039c0 2024-05-04T21:20:58.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7255d9b640 1 -- 172.21.15.52:0/1848169177 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f724400b9d0 con 0x7f72580aa0c0 2024-05-04T21:20:58.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f724ffff640 1 --2- 172.21.15.52:0/1848169177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7244004810 0x7f7244004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f725659c640 1 --2- 172.21.15.52:0/1848169177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72440039c0 0x7f7244003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f7256d9d640 1 --2- 172.21.15.52:0/1848169177 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72580aa0c0 0x7f72440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f724ffff640 1 -- 172.21.15.52:0/1848169177 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72580aa0c0 msgr2=0x7f72440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:58.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f724ffff640 1 --2- 172.21.15.52:0/1848169177 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72580aa0c0 0x7f72440051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f724ffff640 1 -- 172.21.15.52:0/1848169177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72440039c0 msgr2=0x7f7244003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T21:20:58.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f724ffff640 1 --2- 172.21.15.52:0/1848169177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72440039c0 0x7f7244003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.735+0000 7f724ffff640 1 -- 172.21.15.52:0/1848169177 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7244003530 con 0x7f7244004810 2024-05-04T21:20:58.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f724ffff640 1 --2- 172.21.15.52:0/1848169177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7244004810 0x7f7244004c90 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f7240009ae0 tx=0x7f724001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e010af594e90f2b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:58.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f7255d9b640 1 -- 172.21.15.52:0/1848169177 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72400b2070 con 0x7f7244004810 2024-05-04T21:20:58.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f7255d9b640 1 -- 172.21.15.52:0/1848169177 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f72400a5580 con 0x7f7244004810 2024-05-04T21:20:58.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f7255d9b640 1 -- 172.21.15.52:0/1848169177 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72400096b0 con 0x7f7244004810 2024-05-04T21:20:58.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725ccf2640 1 -- 172.21.15.52:0/1848169177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7244004810 msgr2=0x7f7244004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725ccf2640 1 --2- 172.21.15.52:0/1848169177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7244004810 0x7f7244004c90 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f7240009ae0 tx=0x7f724001ca70 comp rx=0 tx=0).stop 2024-05-04T21:20:58.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f7256d9d640 1 -- 172.21.15.52:0/1848169177 reap_dead start 2024-05-04T21:20:58.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725ccf2640 1 -- 172.21.15.52:0/1848169177 shutdown_connections 2024-05-04T21:20:58.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725ccf2640 1 -- 172.21.15.52:0/1848169177 >> 172.21.15.52:0/1848169177 conn(0x7f72580b14a0 msgr2=0x7f72580b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:58.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725ccf2640 1 -- 172.21.15.52:0/1848169177 shutdown_connections 2024-05-04T21:20:58.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725ccf2640 1 -- 172.21.15.52:0/1848169177 wait complete. 2024-05-04T21:20:58.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725ccf2640 1 Processor -- start 2024-05-04T21:20:58.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725ccf2640 1 -- start start 2024-05-04T21:20:58.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725ccf2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7258146d00 0x7f72581e1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725ccf2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72581e1dc0 0x7f72580b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725ccf2640 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72581470e0 0x7f72580b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725ccf2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f725815c9f0 con 0x7f7258146d00 2024-05-04T21:20:58.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725ccf2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f725815c870 con 0x7f72581e1dc0 2024-05-04T21:20:58.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725ccf2640 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 -- 0x7f725806cd60 con 0x7f72581470e0 2024-05-04T21:20:58.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.739+0000 7f725659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7258146d00 0x7f72581e1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f725659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7258146d00 0x7f72581e1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43774/0 (socket says 172.21.15.52:43774) 2024-05-04T21:20:58.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f725659c640 1 -- 172.21.15.52:0/266147451 learned_addr learned my addr 172.21.15.52:0/266147451 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-04T21:20:58.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f7256d9d640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72581470e0 0x7f72580b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f724ffff640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72581e1dc0 0x7f72580b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f725659c640 1 -- 172.21.15.52:0/266147451 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72581470e0 msgr2=0x7f72580b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f725659c640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72581470e0 0x7f72580b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f725659c640 1 -- 172.21.15.52:0/266147451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72581e1dc0 msgr2=0x7f72580b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f725659c640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72581e1dc0 0x7f72580b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f725659c640 1 -- 172.21.15.52:0/266147451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72580717d0 con 0x7f7258146d00 2024-05-04T21:20:58.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f7256d9d640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72581470e0 0x7f72580b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T21:20:58.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f725659c640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7258146d00 0x7f72581e1880 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f7248011660 tx=0x7f7248011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:58.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f724effd640 1 -- 172.21.15.52:0/266147451 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724800f9c0 con 0x7f7258146d00 2024-05-04T21:20:58.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f724effd640 1 -- 172.21.15.52:0/266147451 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f724800fb60 con 0x7f7258146d00 2024-05-04T21:20:58.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f725ccf2640 1 -- 172.21.15.52:0/266147451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7258070730 con 0x7f7258146d00 2024-05-04T21:20:58.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f724effd640 1 -- 172.21.15.52:0/266147451 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248015460 con 0x7f7258146d00 2024-05-04T21:20:58.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f725ccf2640 1 -- 172.21.15.52:0/266147451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7258070cf0 con 0x7f7258146d00 2024-05-04T21:20:58.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f725ccf2640 1 -- 172.21.15.52:0/266147451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7258062e50 con 0x7f7258146d00 2024-05-04T21:20:58.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f724effd640 1 -- 172.21.15.52:0/266147451 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7248015600 con 0x7f7258146d00 2024-05-04T21:20:58.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f724effd640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f72380678f0 0x7f7238069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T21:20:58.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f724effd640 1 -- 172.21.15.52:0/266147451 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f7248086380 con 0x7f7258146d00 2024-05-04T21:20:58.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f724ffff640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f72380678f0 0x7f7238069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T21:20:58.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.743+0000 7f724ffff640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f72380678f0 0x7f7238069db0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f72400af000 tx=0x7f7240008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T21:20:58.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.747+0000 7f724effd640 1 -- 172.21.15.52:0/266147451 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7248019030 con 0x7f7258146d00 2024-05-04T21:20:58.920 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:20:58.922 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:20:58.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.939+0000 7f725ccf2640 1 -- 172.21.15.52:0/266147451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7258075320 con 0x7f7258146d00 2024-05-04T21:20:58.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.939+0000 7f724effd640 1 -- 172.21.15.52:0/266147451 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3872 (secure 0 0 0) 0x7f72480571c0 con 0x7f7258146d00 2024-05-04T21:20:58.943 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-04T21:20:58.944 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2438654614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2438654614},{"type":"v1","addr":"172.21.15.111:6835","nonce":2438654614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/796226117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":796226117},{"type":"v1","addr":"172.21.15.52:6835","nonce":796226117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3866938727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3866938727},{"type":"v1","addr":"172.21.15.111:6833","nonce":3866938727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":7274,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/2308675296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":2308675296},{"type":"v1","addr":"172.21.15.52:6837","nonce":2308675296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-05-04T21:20:58.944 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 18 2024-05-04T21:20:58.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.939+0000 7f725ccf2640 1 -- 172.21.15.52:0/266147451 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f72380678f0 msgr2=0x7f7238069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.939+0000 7f725ccf2640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f72380678f0 0x7f7238069db0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f72400af000 tx=0x7f7240008040 comp rx=0 tx=0).stop 2024-05-04T21:20:58.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.939+0000 7f725ccf2640 1 -- 172.21.15.52:0/266147451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7258146d00 msgr2=0x7f72581e1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T21:20:58.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.939+0000 7f725ccf2640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7258146d00 0x7f72581e1880 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f7248011660 tx=0x7f7248011b20 comp rx=0 tx=0).stop 2024-05-04T21:20:58.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.939+0000 7f725ccf2640 1 -- 172.21.15.52:0/266147451 shutdown_connections 2024-05-04T21:20:58.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.939+0000 7f725ccf2640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.52:6832/405834661,v1:172.21.15.52:6833/405834661] conn(0x7f72380678f0 0x7f7238069db0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.939+0000 7f725ccf2640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72581470e0 0x7f72580b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.939+0000 7f725ccf2640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72581e1dc0 0x7f72580b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.939+0000 7f725ccf2640 1 --2- 172.21.15.52:0/266147451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7258146d00 0x7f72581e1880 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T21:20:58.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.939+0000 7f725ccf2640 1 -- 172.21.15.52:0/266147451 >> 172.21.15.52:0/266147451 conn(0x7f72580b14a0 msgr2=0x7f7258158ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T21:20:58.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.939+0000 7f725ccf2640 1 -- 172.21.15.52:0/266147451 shutdown_connections 2024-05-04T21:20:58.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-04T21:20:58.939+0000 7f725ccf2640 1 -- 172.21.15.52:0/266147451 wait complete. 2024-05-04T21:20:58.961 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-04T21:20:58.961 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-04T21:20:58.961 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-04T21:20:58.962 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:21:01.689 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-05-04T21:21:05.062 INFO:tasks.ceph.mds.a:Stopped 2024-05-04T21:21:05.063 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-04T21:21:05.063 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:21:08.201 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-05-04T21:21:08.202 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-05-04T21:21:11.165 INFO:tasks.ceph.mds.c:Stopped 2024-05-04T21:21:11.165 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-04T21:21:11.166 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:21:14.612 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-05-04T21:21:14.613 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-05-04T21:21:17.268 INFO:tasks.ceph.mds.b:Stopped 2024-05-04T21:21:17.268 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-04T21:21:17.268 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:21:17.299 INFO:tasks.ceph.mds.d:Stopped 2024-05-04T21:21:17.300 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-04T21:21:17.300 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-04T21:21:17.300 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:21:20.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-05-04T21:21:20.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-05-04T21:21:23.401 INFO:tasks.ceph.osd.0:Stopped 2024-05-04T21:21:23.402 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-04T21:21:23.402 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:21:26.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-05-04T21:21:26.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-05-04T21:21:28.998 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:21:29.001 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:21:29.504 INFO:tasks.ceph.osd.1:Stopped 2024-05-04T21:21:29.504 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-04T21:21:29.504 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:21:32.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-05-04T21:21:32.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-05-04T21:21:35.605 INFO:tasks.ceph.osd.2:Stopped 2024-05-04T21:21:35.606 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-04T21:21:35.606 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:21:38.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-05-04T21:21:38.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-05-04T21:21:41.708 INFO:tasks.ceph.osd.3:Stopped 2024-05-04T21:21:41.709 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-04T21:21:41.709 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:21:44.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-05-04T21:21:44.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-05-04T21:21:47.811 INFO:tasks.ceph.osd.4:Stopped 2024-05-04T21:21:47.811 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-04T21:21:47.811 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:21:50.361 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-05-04T21:21:50.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-05-04T21:21:53.913 INFO:tasks.ceph.osd.5:Stopped 2024-05-04T21:21:53.914 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-04T21:21:53.914 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:21:55.967 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-05-04T21:21:55.967 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-05-04T21:21:59.127 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:21:59.130 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:22:00.019 INFO:tasks.ceph.osd.6:Stopped 2024-05-04T21:22:00.019 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-04T21:22:00.019 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:22:01.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-05-04T21:22:01.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-05-04T21:22:06.121 INFO:tasks.ceph.osd.7:Stopped 2024-05-04T21:22:06.121 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-04T21:22:06.122 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-04T21:22:06.122 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:22:06.976 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-05-04T21:22:12.224 INFO:tasks.ceph.mgr.y:Stopped 2024-05-04T21:22:12.224 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-04T21:22:12.224 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:22:17.683 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-05-04T21:22:18.327 INFO:tasks.ceph.mgr.x:Stopped 2024-05-04T21:22:18.327 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-04T21:22:18.327 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-04T21:22:18.327 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:22:22.885 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-05-04T21:22:22.885 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-05-04T21:22:24.429 INFO:tasks.ceph.mon.a:Stopped 2024-05-04T21:22:24.429 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-04T21:22:24.429 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:22:27.987 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-05-04T21:22:27.987 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-05-04T21:22:29.242 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:22:29.244 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:22:30.531 INFO:tasks.ceph.mon.b:Stopped 2024-05-04T21:22:30.532 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-04T21:22:30.532 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T21:22:32.988 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-05-04T21:22:32.988 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-05-04T21:22:36.634 INFO:tasks.ceph.mon.c:Stopped 2024-05-04T21:22:36.634 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-04T21:22:36.634 DEBUG:teuthology.orchestra.run.smithi052:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-04T21:22:36.751 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T21:22:36.751 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-04T21:22:36.997 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T21:22:36.997 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-04T21:22:37.256 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T21:22:37.256 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-04T21:22:37.482 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T21:22:37.483 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-04T21:22:37.742 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T21:22:37.743 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-04T21:22:37.989 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-05-04T21:22:38.086 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T21:22:38.086 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-04T21:22:38.297 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T21:22:38.297 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-04T21:22:38.509 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T21:22:38.509 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-04T21:22:38.738 INFO:tasks.ceph:Archiving mon data... 2024-05-04T21:22:38.740 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278/data/mon.a.tgz 2024-05-04T21:22:38.740 DEBUG:teuthology.orchestra.run.smithi052:> mktemp 2024-05-04T21:22:38.746 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.z2mPDCXpby 2024-05-04T21:22:38.747 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.z2mPDCXpby 2024-05-04T21:22:38.928 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0666 /tmp/tmp.z2mPDCXpby 2024-05-04T21:22:39.045 DEBUG:teuthology.orchestra.remote:smithi052:/tmp/tmp.z2mPDCXpby is 556KB 2024-05-04T21:22:39.108 DEBUG:teuthology.orchestra.run.smithi052:> rm -fr /tmp/tmp.z2mPDCXpby 2024-05-04T21:22:39.115 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278/data/mon.b.tgz 2024-05-04T21:22:39.115 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2024-05-04T21:22:39.121 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.np4XbnmMES 2024-05-04T21:22:39.121 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.np4XbnmMES 2024-05-04T21:22:39.284 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.np4XbnmMES 2024-05-04T21:22:39.398 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.np4XbnmMES is 562KB 2024-05-04T21:22:39.462 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.np4XbnmMES 2024-05-04T21:22:39.467 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278/data/mon.c.tgz 2024-05-04T21:22:39.467 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2024-05-04T21:22:39.516 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.ucdXE4OCTs 2024-05-04T21:22:39.516 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ucdXE4OCTs 2024-05-04T21:22:39.671 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.ucdXE4OCTs 2024-05-04T21:22:39.735 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.ucdXE4OCTs is 562KB 2024-05-04T21:22:39.797 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.ucdXE4OCTs 2024-05-04T21:22:39.803 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-04T21:22:39.803 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-04T21:22:39.806 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-04T21:22:39.915 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-04T21:22:39.941 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-04T21:22:39.941 INFO:tasks.ceph:Archiving crash dumps... 2024-05-04T21:22:39.943 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278/remote/smithi052/crash 2024-05-04T21:22:39.944 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-04T21:22:39.971 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278/remote/smithi111/crash 2024-05-04T21:22:39.972 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-04T21:22:40.004 INFO:tasks.ceph:Shutting down logrotate 2024-05-04T21:22:40.004 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:22:40.016 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-04T21:22:40.096 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-04T21:22:40.138 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-04T21:22:40.150 INFO:tasks.ceph:Compressing logs... 2024-05-04T21:22:40.150 DEBUG:teuthology.orchestra.run.smithi052:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-04T21:22:40.184 DEBUG:teuthology.orchestra.run.smithi111:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-04T21:22:40.213 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.421429.log 2024-05-04T21:22:40.214 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32848.log 2024-05-04T21:22:40.215 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.421429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79069.log 2024-05-04T21:22:40.215 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.32848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423530.log 2024-05-04T21:22:40.216 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.79069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.197157.log 2024-05-04T21:22:40.216 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.223455.log 2024-05-04T21:22:40.217 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.423530.log: /var/log/ceph/ceph-client.admin.197157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.314994.log 2024-05-04T21:22:40.217 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.223455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490582.log 2024-05-04T21:22:40.219 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.314994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.469189.log 2024-05-04T21:22:40.219 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.483900.log 2024-05-04T21:22:40.219 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490582.log: /var/log/ceph/ceph-client.admin.469189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166801.log 2024-05-04T21:22:40.220 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.490582.log.gz 2024-05-04T21:22:40.220 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.483900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102587.log 2024-05-04T21:22:40.220 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.166801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.282464.log 2024-05-04T21:22:40.220 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.102587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35982.log 2024-05-04T21:22:40.221 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.282464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.327336.log 2024-05-04T21:22:40.221 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.303550.log 2024-05-04T21:22:40.233 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27841.log 2024-05-04T21:22:40.233 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27913.log 2024-05-04T21:22:40.233 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.27841.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-04T21:22:40.233 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.27913.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-04T21:22:40.234 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27793.log 2024-05-04T21:22:40.234 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33492.log 2024-05-04T21:22:40.234 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.27793.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-04T21:22:40.234 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33611.log 2024-05-04T21:22:40.234 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.33492.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-04T21:22:40.234 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-04T21:22:40.234 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.3.33611.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-04T21:22:40.234 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27841.log.gz 2024-05-04T21:22:40.235 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-04T21:22:40.235 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27817.log 2024-05-04T21:22:40.235 INFO:teuthology.orchestra.run.smithi111.stderr:gzip/var/log/ceph/ceph.audit.log: -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-04T21:22:40.235 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.27817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27937.log 2024-05-04T21:22:40.235 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-04T21:22:40.235 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.27937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27889.log 2024-05-04T21:22:40.235 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-04T21:22:40.235 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-client.admin.27889.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-04T21:22:40.235 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27961.log 2024-05-04T21:22:40.236 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27865.log 2024-05-04T21:22:40.236 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.27961.log: /var/log/ceph/ceph-client.admin.27865.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27865.log.gz 2024-05-04T21:22:40.236 INFO:teuthology.orchestra.run.smithi111.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27793.log.gz 2024-05-04T21:22:40.236 INFO:teuthology.orchestra.run.smithi111.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27817.log.gz 2024-05-04T21:22:40.252 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.327336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145436.log 2024-05-04T21:22:40.252 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.227650.log 2024-05-04T21:22:40.252 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35982.log.gz 2024-05-04T21:22:40.252 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.303550.log: /var/log/ceph/ceph-client.admin.145436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147075.log 2024-05-04T21:22:40.252 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.227650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.349461.log 2024-05-04T21:22:40.253 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.147075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37379.log 2024-05-04T21:22:40.253 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.349461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.488600.log 2024-05-04T21:22:40.253 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.37379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.400035.log 2024-05-04T21:22:40.253 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.488600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106262.log 2024-05-04T21:22:40.253 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.400035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.231728.log 2024-05-04T21:22:40.253 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.106262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171484.log 2024-05-04T21:22:40.253 INFO:teuthology.orchestra.run.smithi052.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.223455.log.gz 2024-05-04T21:22:40.253 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.231728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.210030.log 2024-05-04T21:22:40.254 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.171484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491171.log 2024-05-04T21:22:40.254 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.210030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192027.log 2024-05-04T21:22:40.254 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.491171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.435787.log 2024-05-04T21:22:40.254 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.192027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84980.log 2024-05-04T21:22:40.254 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.491171.log.gz 2024-05-04T21:22:40.254 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.435787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152197.log 2024-05-04T21:22:40.254 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.84980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.397384.log 2024-05-04T21:22:40.254 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.152197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181416.log 2024-05-04T21:22:40.255 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.397384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.469809.log 2024-05-04T21:22:40.255 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.181416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98922.log 2024-05-04T21:22:40.255 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.469809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.302690.log 2024-05-04T21:22:40.255 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.98922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.426050.log 2024-05-04T21:22:40.255 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.302690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.386890.log 2024-05-04T21:22:40.255 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.426050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36533.log 2024-05-04T21:22:40.255 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.386890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.296590.log 2024-05-04T21:22:40.256 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.418195.log 2024-05-04T21:22:40.256 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.296590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144213.log 2024-05-04T21:22:40.256 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.418195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.210021.log 2024-05-04T21:22:40.256 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.144213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35719.log 2024-05-04T21:22:40.256 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.314994.log.gz 2024-05-04T21:22:40.256 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.210021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147491.log 2024-05-04T21:22:40.256 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.369077.log 2024-05-04T21:22:40.256 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.147491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.383790.log 2024-05-04T21:22:40.257 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.369077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.343351.log 2024-05-04T21:22:40.257 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.383790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176364.log 2024-05-04T21:22:40.257 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.343351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.289006.log 2024-05-04T21:22:40.257 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.176364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.382530.log 2024-05-04T21:22:40.257 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.289006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165922.log 2024-05-04T21:22:40.257 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.225223.log 2024-05-04T21:22:40.257 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.382530.log: /var/log/ceph/ceph-client.admin.165922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160742.log 2024-05-04T21:22:40.257 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.225223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129935.log 2024-05-04T21:22:40.258 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.210030.log.gz 2024-05-04T21:22:40.258 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.160742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.441334.log 2024-05-04T21:22:40.258 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.129935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.231719.log 2024-05-04T21:22:40.258 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.441334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36812.log 2024-05-04T21:22:40.258 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.231719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100952.log 2024-05-04T21:22:40.258 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36812.log: gzip -5 84.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.192027.log.gz /var/log/ceph/ceph-client.admin.188161.log 2024-05-04T21:22:40.258 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-04T21:22:40.258 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.100952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188170.log 2024-05-04T21:22:40.259 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.188161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.382119.log 2024-05-04T21:22:40.259 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.188170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.462007.log 2024-05-04T21:22:40.259 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.382119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36996.log 2024-05-04T21:22:40.259 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.462007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36429.log 2024-05-04T21:22:40.259 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34254.log 2024-05-04T21:22:40.259 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.84980.log.gz 2024-05-04T21:22:40.259 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125483.log 2024-05-04T21:22:40.259 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490911.log 2024-05-04T21:22:40.260 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.486856.log 2024-05-04T21:22:40.260 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.463323.log 2024-05-04T21:22:40.260 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.200053.log 2024-05-04T21:22:40.260 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100555.log 2024-05-04T21:22:40.260 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32973.log 2024-05-04T21:22:40.260 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35641.log 2024-05-04T21:22:40.260 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-04T21:22:40.260 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.488127.log 2024-05-04T21:22:40.261 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.478268.log 2024-05-04T21:22:40.261 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.456179.log 2024-05-04T21:22:40.261 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.348650.log 2024-05-04T21:22:40.261 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.488609.log 2024-05-04T21:22:40.261 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156843.log 2024-05-04T21:22:40.261 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161177.log 2024-05-04T21:22:40.261 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.453741.log 2024-05-04T21:22:40.261 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490426.log 2024-05-04T21:22:40.262 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35615.log 2024-05-04T21:22:40.262 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.380906.log 2024-05-04T21:22:40.262 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.228868.log 2024-05-04T21:22:40.262 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95990.log 2024-05-04T21:22:40.262 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170663.log 2024-05-04T21:22:40.262 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.459827.log 2024-05-04T21:22:40.262 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.426520.log 2024-05-04T21:22:40.263 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102994.log 2024-05-04T21:22:40.263 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.394042.log 2024-05-04T21:22:40.263 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36221.log 2024-05-04T21:22:40.263 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.380499.log 2024-05-04T21:22:40.263 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.349890.log 2024-05-04T21:22:40.263 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.199249.log 2024-05-04T21:22:40.263 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.452509.log 2024-05-04T21:22:40.263 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.281641.log 2024-05-04T21:22:40.264 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.322470.log 2024-05-04T21:22:40.264 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.369497.log 2024-05-04T21:22:40.264 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.459014.log 2024-05-04T21:22:40.264 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.301851.log 2024-05-04T21:22:40.264 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.396536.log 2024-05-04T21:22:40.264 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146262.log 2024-05-04T21:22:40.264 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.425211.log 2024-05-04T21:22:40.264 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.329787.log 2024-05-04T21:22:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92257.log 2024-05-04T21:22:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.375261.log 2024-05-04T21:22:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.291963.log 2024-05-04T21:22:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34976.log 2024-05-04T21:22:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491249.log 2024-05-04T21:22:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.416397.log 2024-05-04T21:22:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.217307.log 2024-05-04T21:22:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34150.log 2024-05-04T21:22:40.266 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.298616.log 2024-05-04T21:22:40.266 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.363306.log 2024-05-04T21:22:40.267 INFO:teuthology.orchestra.run.smithi111.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-04T21:22:40.281 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.430401.log 2024-05-04T21:22:40.281 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.457796.log 2024-05-04T21:22:40.281 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111819.log 2024-05-04T21:22:40.281 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359393.log 2024-05-04T21:22:40.281 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.418657.log 2024-05-04T21:22:40.282 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.294920.log 2024-05-04T21:22:40.282 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.328820.log 2024-05-04T21:22:40.282 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.215677.log 2024-05-04T21:22:40.282 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.398274.log 2024-05-04T21:22:40.282 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.330257.log 2024-05-04T21:22:40.282 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.452518.log 2024-05-04T21:22:40.282 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84329.log 2024-05-04T21:22:40.282 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149243.log 2024-05-04T21:22:40.283 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.321242.log 2024-05-04T21:22:40.283 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.216083.log 2024-05-04T21:22:40.283 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35080.log 2024-05-04T21:22:40.283 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108037.log 2024-05-04T21:22:40.283 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188986.log 2024-05-04T21:22:40.283 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120958.log 2024-05-04T21:22:40.283 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112651.log 2024-05-04T21:22:40.283 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.465478.log 2024-05-04T21:22:40.284 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.325160.log 2024-05-04T21:22:40.284 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.228462.log 2024-05-04T21:22:40.284 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.386079.log 2024-05-04T21:22:40.284 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93922.log 2024-05-04T21:22:40.284 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.470643.log 2024-05-04T21:22:40.284 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186537.log 2024-05-04T21:22:40.284 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.355637.log 2024-05-04T21:22:40.284 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.289402.log 2024-05-04T21:22:40.285 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.230907.log 2024-05-04T21:22:40.285 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.342459.log 2024-05-04T21:22:40.285 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34072.log 2024-05-04T21:22:40.285 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.451697.log 2024-05-04T21:22:40.285 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.291972.log 2024-05-04T21:22:40.285 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32947.log 2024-05-04T21:22:40.285 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.408487.log 2024-05-04T21:22:40.285 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168096.log 2024-05-04T21:22:40.286 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27566.log 2024-05-04T21:22:40.286 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.409705.log 2024-05-04T21:22:40.286 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.303088.log 2024-05-04T21:22:40.286 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.312223.log 2024-05-04T21:22:40.286 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.243177.log 2024-05-04T21:22:40.286 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190253.log 2024-05-04T21:22:40.286 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101766.log 2024-05-04T21:22:40.287 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.476991.log 2024-05-04T21:22:40.287 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.367045.log 2024-05-04T21:22:40.287 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.385677.log 2024-05-04T21:22:40.287 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157646.log 2024-05-04T21:22:40.287 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490478.log 2024-05-04T21:22:40.287 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.397802.log 2024-05-04T21:22:40.287 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359650.log 2024-05-04T21:22:40.287 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-04T21:22:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.219760.log 2024-05-04T21:22:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174317.log 2024-05-04T21:22:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.415990.log 2024-05-04T21:22:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.426059.log 2024-05-04T21:22:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.357359.log 2024-05-04T21:22:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.223035.log 2024-05-04T21:22:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37405.log 2024-05-04T21:22:40.289 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.432439.log 2024-05-04T21:22:40.289 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.434510.log 2024-05-04T21:22:40.289 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.286142.log 2024-05-04T21:22:40.289 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.473577.log 2024-05-04T21:22:40.289 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.480414.log 2024-05-04T21:22:40.289 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92673.log 2024-05-04T21:22:40.289 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.357803.log 2024-05-04T21:22:40.289 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.227659.log 2024-05-04T21:22:40.290 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151346.log 2024-05-04T21:22:40.290 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97688.log 2024-05-04T21:22:40.290 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.485159.log 2024-05-04T21:22:40.290 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35132.log 2024-05-04T21:22:40.290 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.361637.log 2024-05-04T21:22:40.290 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.303960.log 2024-05-04T21:22:40.290 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.436184.log 2024-05-04T21:22:40.291 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106715.log 2024-05-04T21:22:40.291 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175548.log 2024-05-04T21:22:40.291 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194705.log 2024-05-04T21:22:40.291 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491223.log 2024-05-04T21:22:40.291 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78672.log 2024-05-04T21:22:40.291 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105842.log 2024-05-04T21:22:40.291 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.468442.log 2024-05-04T21:22:40.291 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.318781.log 2024-05-04T21:22:40.292 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121833.log 2024-05-04T21:22:40.292 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.395318.log 2024-05-04T21:22:40.292 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44678.log 2024-05-04T21:22:40.292 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.358209.log 2024-05-04T21:22:40.292 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171881.log 2024-05-04T21:22:40.292 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176771.log 2024-05-04T21:22:40.292 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195518.log 2024-05-04T21:22:40.292 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36689.log 2024-05-04T21:22:40.293 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.433677.log 2024-05-04T21:22:40.293 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34768.log 2024-05-04T21:22:40.293 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.479556.log 2024-05-04T21:22:40.293 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359345.log 2024-05-04T21:22:40.293 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128843.log 2024-05-04T21:22:40.293 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168977.log 2024-05-04T21:22:40.293 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.362895.log 2024-05-04T21:22:40.293 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181822.log 2024-05-04T21:22:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490686.log 2024-05-04T21:22:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.451706.log 2024-05-04T21:22:40.308 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33292.log 2024-05-04T21:22:40.308 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.285722.log 2024-05-04T21:22:40.308 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.410166.log 2024-05-04T21:22:40.309 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.350310.log 2024-05-04T21:22:40.309 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.455773.log 2024-05-04T21:22:40.309 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.404589.log 2024-05-04T21:22:40.309 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.370723.log 2024-05-04T21:22:40.309 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.292815.log 2024-05-04T21:22:40.309 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.431222.log 2024-05-04T21:22:40.309 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176762.log 2024-05-04T21:22:40.309 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.290221.log 2024-05-04T21:22:40.310 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491015.log 2024-05-04T21:22:40.310 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.319195.log 2024-05-04T21:22:40.310 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490731.log 2024-05-04T21:22:40.310 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122288.log 2024-05-04T21:22:40.310 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.206778.log 2024-05-04T21:22:40.310 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195915.log 2024-05-04T21:22:40.310 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118877.log 2024-05-04T21:22:40.311 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490911.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.327336.log.gzgzip 2024-05-04T21:22:40.311 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.185675.log 2024-05-04T21:22:40.311 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.486856.log: /var/log/ceph/ceph-client.admin.125483.log: /var/log/ceph/ceph-client.admin.463323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.385037.log 2024-05-04T21:22:40.311 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175131.log 2024-05-04T21:22:40.311 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125474.log 2024-05-04T21:22:40.311 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177178.log 2024-05-04T21:22:40.311 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.346280.log 2024-05-04T21:22:40.311 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.217727.log 2024-05-04T21:22:40.312 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.211651.log 2024-05-04T21:22:40.312 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181425.log 2024-05-04T21:22:40.312 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.343342.log 2024-05-04T21:22:40.312 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183867.log 2024-05-04T21:22:40.312 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.373213.log 2024-05-04T21:22:40.312 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105851.log 2024-05-04T21:22:40.312 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.376928.log 2024-05-04T21:22:40.312 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35349.log 2024-05-04T21:22:40.313 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185269.log 2024-05-04T21:22:40.313 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.297400.log 2024-05-04T21:22:40.313 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154393.log 2024-05-04T21:22:40.313 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79487.log 2024-05-04T21:22:40.313 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.378146.log 2024-05-04T21:22:40.313 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.222624.log 2024-05-04T21:22:40.313 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.308048.log 2024-05-04T21:22:40.313 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162878.log 2024-05-04T21:22:40.314 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.427341.log 2024-05-04T21:22:40.314 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.239299.log 2024-05-04T21:22:40.314 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35297.log 2024-05-04T21:22:40.314 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146664.log 2024-05-04T21:22:40.314 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33691.log 2024-05-04T21:22:40.314 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.327842.log 2024-05-04T21:22:40.314 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359384.log 2024-05-04T21:22:40.315 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173099.log 2024-05-04T21:22:40.315 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.446769.log 2024-05-04T21:22:40.315 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.241034.log 2024-05-04T21:22:40.315 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116759.log 2024-05-04T21:22:40.315 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.199655.log 2024-05-04T21:22:40.315 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.400488.log 2024-05-04T21:22:40.315 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36060.log 2024-05-04T21:22:40.315 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120552.log 2024-05-04T21:22:40.316 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112642.log 2024-05-04T21:22:40.316 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490556.log 2024-05-04T21:22:40.316 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.448318.log 2024-05-04T21:22:40.316 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104207.log 2024-05-04T21:22:40.316 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169843.log 2024-05-04T21:22:40.316 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.197166.log 2024-05-04T21:22:40.316 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.287771.log 2024-05-04T21:22:40.316 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.225629.log 2024-05-04T21:22:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.315400.log 2024-05-04T21:22:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127733.log 2024-05-04T21:22:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35454.log 2024-05-04T21:22:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.485168.log 2024-05-04T21:22:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.364158.log 2024-05-04T21:22:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.384195.log 2024-05-04T21:22:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36034.log 2024-05-04T21:22:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125894.log 2024-05-04T21:22:40.318 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160733.log 2024-05-04T21:22:40.318 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84338.log 2024-05-04T21:22:40.318 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.242358.log 2024-05-04T21:22:40.318 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.480841.log 2024-05-04T21:22:40.318 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.395309.log 2024-05-04T21:22:40.318 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.362459.log 2024-05-04T21:22:40.319 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159458.log 2024-05-04T21:22:40.319 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169413.log 2024-05-04T21:22:40.319 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110948.log 2024-05-04T21:22:40.319 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153917.log 2024-05-04T21:22:40.319 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.473982.log 2024-05-04T21:22:40.319 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.345873.log 2024-05-04T21:22:40.319 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35506.log 2024-05-04T21:22:40.319 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.404177.log 2024-05-04T21:22:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92248.log 2024-05-04T21:22:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.412359.log 2024-05-04T21:22:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.406349.log 2024-05-04T21:22:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.438867.log 2024-05-04T21:22:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112225.log 2024-05-04T21:22:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.365380.log 2024-05-04T21:22:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.205903.log 2024-05-04T21:22:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.431213.log 2024-05-04T21:22:40.321 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.210848.log 2024-05-04T21:22:40.452 INFO:teuthology.orchestra.run.smithi111.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.1.33492.log.gz 2024-05-04T21:22:40.453 INFO:teuthology.orchestra.run.smithi111.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.3.33611.log.gz 2024-05-04T21:22:40.453 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27913.log.gz 2024-05-04T21:22:40.453 INFO:teuthology.orchestra.run.smithi111.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27937.log.gz 2024-05-04T21:22:40.453 INFO:teuthology.orchestra.run.smithi111.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27961.log.gz 2024-05-04T21:22:40.629 INFO:teuthology.orchestra.run.smithi111.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27889.log.gz 2024-05-04T21:22:40.817 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491041.log 2024-05-04T21:22:40.818 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490757.log 2024-05-04T21:22:40.818 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.229690.log 2024-05-04T21:22:40.818 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.482556.log 2024-05-04T21:22:40.818 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.209610.log 2024-05-04T21:22:40.818 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179771.log 2024-05-04T21:22:40.818 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.197573.log 2024-05-04T21:22:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.439264.log 2024-05-04T21:22:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.328322.log 2024-05-04T21:22:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.449801.log 2024-05-04T21:22:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.243583.log 2024-05-04T21:22:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35825.log 2024-05-04T21:22:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.438858.log 2024-05-04T21:22:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.44678.log: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.469809.log.gz 2024-05-04T21:22:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.488127.log: /var/log/ceph/ceph-client.admin.35825.log: /var/log/ceph/ceph-client.admin.478268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.458202.log 2024-05-04T21:22:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.456179.log: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.349461.log.gz 2024-05-04T21:22:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.488609.log: /var/log/ceph/ceph-client.admin.453741.log: /var/log/ceph/ceph-client.admin.438858.log: /var/log/ceph/ceph-client.admin.490426.log: /var/log/ceph/ceph-client.admin.380906.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.469189.log.gz 2024-05-04T21:22:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.244412.log 2024-05-04T21:22:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.459827.log: /var/log/ceph/ceph-client.admin.426520.log: /var/log/ceph/ceph-client.admin.394042.log: /var/log/ceph/ceph-client.admin.380499.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.282464.log.gz 2024-05-04T21:22:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.452509.log: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.488600.log.gz 2024-05-04T21:22:40.820 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.490426.log.gz/var/log/ceph/ceph-client.admin.369497.log: 2024-05-04T21:22:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.459014.log: /var/log/ceph/ceph-client.admin.396536.log: /var/log/ceph/ceph-client.admin.425211.log: /var/log/ceph/ceph-client.admin.375261.log: /var/log/ceph/ceph-client.admin.458202.log: /var/log/ceph/ceph-client.admin.491249.log: /var/log/ceph/ceph-client.admin.416397.log: gzip/var/log/ceph/ceph-client.admin.363306.log: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.166801.log.gz 2024-05-04T21:22:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.430401.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.483891.log 2024-05-04T21:22:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.457796.log: /var/log/ceph/ceph-client.admin.418657.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.181416.log.gz 2024-05-04T21:22:40.821 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.398274.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.491249.log.gz 2024-05-04T21:22:40.821 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.452518.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44678.log.gz 2024-05-04T21:22:40.821 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.465478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.288173.log 2024-05-04T21:22:40.821 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.386079.log: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.100952.log.gz 2024-05-04T21:22:40.821 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.98922.log.gz 2024-05-04T21:22:40.821 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.470643.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.102587.log.gz 2024-05-04T21:22:40.821 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.451697.log: /var/log/ceph/ceph-client.admin.408487.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.397384.log.gz 2024-05-04T21:22:40.821 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.409705.log: /var/log/ceph/ceph-client.admin.483891.log: /var/log/ceph/ceph-client.admin.476991.log: /var/log/ceph/ceph-client.admin.367045.log: /var/log/ceph/ceph-client.admin.385677.log: /var/log/ceph/ceph-client.admin.490478.log: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.421429.log.gz 2024-05-04T21:22:40.822 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.423530.log.gz 2024-05-04T21:22:40.822 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35825.log.gz 2024-05-04T21:22:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183456.log/var/log/ceph/ceph-client.admin.397802.log: 2024-05-04T21:22:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.288173.log: /var/log/ceph/ceph-client.admin.359650.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.435787.log.gz 2024-05-04T21:22:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-osd.0.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.490478.log.gz 2024-05-04T21:22:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.415990.log: /var/log/ceph/ceph-client.admin.426059.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.382530.log.gz 2024-05-04T21:22:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.384916.log 2024-05-04T21:22:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.432439.log: /var/log/ceph/ceph-client.admin.434510.log: /var/log/ceph/ceph-client.admin.473577.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.383790.log.gz 2024-05-04T21:22:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.480414.log: /var/log/ceph/ceph-client.admin.485159.log: 83.0% -- replaced with /var/log/ceph/ceph-client.admin.386890.log.gz 2024-05-04T21:22:40.823 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.361637.log: /var/log/ceph/ceph-client.admin.436184.log: 80.7%/var/log/ceph/ceph-client.admin.491223.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.147075.log.gz 2024-05-04T21:22:40.823 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.145445.log 2024-05-04T21:22:40.823 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.384916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.457390.log 2024-05-04T21:22:40.823 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.491223.log.gz 2024-05-04T21:22:40.823 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.147491.log.gz/var/log/ceph/ceph-client.admin.468442.log: 2024-05-04T21:22:40.823 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.144213.log.gz 2024-05-04T21:22:40.823 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.395318.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.145436.log.gz 2024-05-04T21:22:40.823 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158101.log 2024-05-04T21:22:40.824 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.433677.log: /var/log/ceph/ceph-client.admin.479556.log: /var/log/ceph/ceph-client.admin.362895.log: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.129935.log.gz 2024-05-04T21:22:40.824 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.165922.log.gz/var/log/ceph/ceph-client.admin.490686.log: 2024-05-04T21:22:40.824 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.451706.log: /var/log/ceph/ceph-client.admin.410166.log: /var/log/ceph/ceph-client.admin.457390.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.171484.log.gz 2024-05-04T21:22:40.824 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.455773.log: /var/log/ceph/ceph-client.admin.404589.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.490686.log.gz 2024-05-04T21:22:40.824 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.370723.log: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.227650.log.gz 2024-05-04T21:22:40.824 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.158101.log: /var/log/ceph/ceph-client.admin.431222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37249.log 2024-05-04T21:22:40.824 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3%/var/log/ceph/ceph-client.admin.491015.log: -- replaced with /var/log/ceph/ceph-client.admin.35719.log.gz 2024-05-04T21:22:40.824 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490731.log: /var/log/ceph/ceph-client.admin.200053.log: /var/log/ceph/ceph-client.admin.199249.log: /var/log/ceph/ceph-client.admin.32973.log: /var/log/ceph/ceph-client.admin.102994.log: /var/log/ceph/ceph-client.admin.281641.log: /var/log/ceph/ceph-client.admin.170663.log: /var/log/ceph/ceph-client.admin.37405.log: /var/log/ceph/ceph-client.admin.188986.log: /var/log/ceph/ceph-client.admin.286142.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.491015.log.gz 2024-05-04T21:22:40.824 INFO:teuthology.orchestra.run.smithi052.stderr: 87.6% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.490911.log.gz 2024-05-04T21:22:40.825 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.490731.log.gz 2024-05-04T21:22:40.825 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.95990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.244403.log 2024-05-04T21:22:40.825 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.112651.log: /var/log/ceph/ceph-client.admin.92673.log: /var/log/ceph/ceph-client.admin.151346.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.382119.log.gz 2024-05-04T21:22:40.825 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.228868.log: /var/log/ceph/ceph-client.admin.37249.log: /var/log/ceph/ceph-client.admin.34072.log: /var/log/ceph/ceph-client.admin.34150.log: /var/log/ceph/ceph-client.admin.227659.log: /var/log/ceph/ceph-client.admin.358209.log: /var/log/ceph/ceph-client.admin.35615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34794.log 2024-05-04T21:22:40.825 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.355637.log: /var/log/ceph/ceph-client.admin.190253.log: /var/log/ceph/ceph-client.admin.97688.log: /var/log/ceph/ceph-client.admin.342459.log: /var/log/ceph/ceph-client.admin.349890.log: /var/log/ceph/ceph-client.admin.176771.log: /var/log/ceph/ceph-client.admin.230907.log: gzip -5/var/log/ceph/ceph-client.admin.161177.log: --verbose -- /var/log/ceph/ceph-client.admin.433668.log 2024-05-04T21:22:40.825 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35132.log: /var/log/ceph/ceph-client.admin.32947.log: /var/log/ceph/ceph-client.admin.322470.log: /var/log/ceph/ceph-client.admin.128843.log: /var/log/ceph/ceph-client.admin.36689.log: /var/log/ceph/ceph-client.admin.156843.log: /var/log/ceph/ceph-client.admin.350310.log: /var/log/ceph/ceph-client.admin.195518.log: /var/log/ceph/ceph-client.admin.312223.log: /var/log/ceph/ceph-client.admin.301851.log: /var/log/ceph/ceph-client.admin.348650.log: /var/log/ceph/ceph-client.admin.34768.log: gzip -5/var/log/ceph/ceph-client.admin.290221.log: --verbose -- /var/log/ceph/ceph-client.admin.216074.log 2024-05-04T21:22:40.825 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.171881.log: /var/log/ceph/ceph-client.admin.357359.log: /var/log/ceph/ceph-client.admin.146262.log: /var/log/ceph/ceph-client.admin.35641.log: /var/log/ceph/ceph-client.admin.319195.log: /var/log/ceph/ceph-client.admin.359345.log: /var/log/ceph/ceph-client.admin.329787.log: /var/log/ceph/ceph-client.admin.433668.log: /var/log/ceph/ceph-client.admin.194705.log: /var/log/ceph/ceph-client.admin.100555.log: /var/log/ceph/ceph-client.admin.122288.log: /var/log/ceph/ceph-client.admin.168977.log: /var/log/ceph/ceph-client.admin.92257.log: gzip/var/log/ceph/ceph-client.admin.195915.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.322045.log 2024-05-04T21:22:40.825 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.78672.log: /var/log/ceph/ceph-client.admin.36221.log: /var/log/ceph/ceph-client.admin.181822.log: /var/log/ceph/ceph-client.admin.291963.log: /var/log/ceph/ceph-client.admin.121833.log: /var/log/ceph/ceph-client.admin.303960.log: /var/log/ceph/ceph-client.admin.285722.log: /var/log/ceph/ceph-client.admin.33292.log: /var/log/ceph/ceph-client.admin.93922.log: /var/log/ceph/ceph-client.admin.228462.log: /var/log/ceph/ceph-client.admin.157646.log: gzip/var/log/ceph/ceph-client.admin.292815.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.241960.log 2024-05-04T21:22:40.826 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.101766.log: /var/log/ceph/ceph-client.admin.219760.log: /var/log/ceph/ceph-client.admin.325160.log: /var/log/ceph/ceph-client.admin.106715.log: /var/log/ceph/ceph-client.admin.176762.log: /var/log/ceph/ceph-client.admin.243177.log: /var/log/ceph/ceph-client.admin.175548.log: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.197157.log.gz 2024-05-04T21:22:40.826 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.206778.log: /var/log/ceph/ceph-client.admin.120958.log: /var/log/ceph/ceph-client.admin.357803.log: /var/log/ceph/ceph-client.admin.303088.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.27566.log: /var/log/ceph/ceph-client.admin.355222.log 2024-05-04T21:22:40.826 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.223035.log: /var/log/ceph/ceph-client.admin.105842.log: /var/log/ceph/ceph-client.admin.108037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.450885.log 2024-05-04T21:22:40.826 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36918.log 2024-05-04T21:22:40.826 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86246.log 2024-05-04T21:22:40.826 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.375270.log 2024-05-04T21:22:40.826 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37022.log 2024-05-04T21:22:40.827 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.244403.log: /var/log/ceph/ceph-client.admin.243583.log: /var/log/ceph/ceph-client.admin.168096.log: /var/log/ceph/ceph-client.admin.174317.log: /var/log/ceph/ceph-client.admin.318781.log: /var/log/ceph/ceph-client.admin.35080.log: /var/log/ceph/ceph-client.admin.375270.log: /var/log/ceph/ceph-client.admin.294920.log: /var/log/ceph/ceph-client.admin.291972.log: /var/log/ceph/ceph-client.admin.298616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.234259.log 2024-05-04T21:22:40.827 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.328820.log: /var/log/ceph/ceph-client.admin.217307.log: /var/log/ceph/ceph-client.admin.111819.log: /var/log/ceph/ceph-client.admin.215677.log: /var/log/ceph/ceph-client.admin.289402.log: /var/log/ceph/ceph-client.admin.34976.log: /var/log/ceph/ceph-client.admin.359393.log: /var/log/ceph/ceph-client.admin.330257.log: /var/log/ceph/ceph-client.admin.385037.log: /var/log/ceph/ceph-client.admin.186537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33916.log 2024-05-04T21:22:40.827 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.84329.log: /var/log/ceph/ceph-client.admin.321242.log: /var/log/ceph/ceph-client.admin.149243.log: /var/log/ceph/ceph-client.admin.216083.log: /var/log/ceph/ceph-client.admin.373213.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.225223.log.gz 2024-05-04T21:22:40.827 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.234259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.306011.log 2024-05-04T21:22:40.827 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.376928.log: /var/log/ceph/ceph-client.admin.378146.log: /var/log/ceph/ceph-client.admin.33916.log: /var/log/ceph/ceph-client.admin.427341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112216.log 2024-05-04T21:22:40.827 INFO:teuthology.orchestra.run.smithi052.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.303550.log.gz 2024-05-04T21:22:40.827 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.302690.log.gz 2024-05-04T21:22:40.827 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.446769.log: /var/log/ceph/ceph-client.admin.306011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.345882.log 2024-05-04T21:22:40.828 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.400488.log: /var/log/ceph/ceph-client.admin.490556.log: /var/log/ceph/ceph-client.admin.448318.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.296590.log.gz/var/log/ceph/ceph-client.admin.485168.log: 2024-05-04T21:22:40.828 INFO:teuthology.orchestra.run.smithi052.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.210021.log.gz 2024-05-04T21:22:40.828 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.112216.log: /var/log/ceph/ceph-client.admin.364158.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.490556.log.gz 2024-05-04T21:22:40.828 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.384195.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.79069.log.gz 2024-05-04T21:22:40.828 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.345882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.313172.log 2024-05-04T21:22:40.828 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.480841.log: /var/log/ceph/ceph-client.admin.395309.log: /var/log/ceph/ceph-client.admin.362459.log: /var/log/ceph/ceph-client.admin.473982.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.369077.log.gz 2024-05-04T21:22:40.828 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.404177.log: /var/log/ceph/ceph-client.admin.412359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423945.log 2024-05-04T21:22:40.828 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.406349.log: /var/log/ceph/ceph-client.admin.438867.log: /var/log/ceph/ceph-client.admin.365380.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.125483.log.gz 2024-05-04T21:22:40.828 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.431213.log: /var/log/ceph/ceph-client.admin.313172.log: /var/log/ceph/ceph-client.admin.491041.log: /var/log/ceph/ceph-client.admin.490757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.443812.log 2024-05-04T21:22:40.829 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.490757.log.gz 2024-05-04T21:22:40.829 INFO:teuthology.orchestra.run.smithi052.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.491041.log.gz 2024-05-04T21:22:40.829 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-04T21:22:40.829 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.423945.log: /var/log/ceph/ceph-client.admin.482556.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.289006.log.gz 2024-05-04T21:22:40.829 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.443812.log: /var/log/ceph/ceph-client.admin.439264.log: /var/log/ceph/ceph-client.admin.449801.log: /var/log/ceph/ceph-client.admin.118877.log: /var/log/ceph/ceph-client.admin.86246.log: /var/log/ceph/ceph-client.admin.211651.log: /var/log/ceph/ceph-client.admin.185675.log: /var/log/ceph/ceph-client.admin.183867.log: /var/log/ceph/ceph-client.admin.162878.log: /var/log/ceph/ceph-client.admin.36060.log: /var/log/ceph/ceph-client.admin.125474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.308057.log 2024-05-04T21:22:40.829 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.110948.log: /var/log/ceph/ceph-client.admin.35349.log: /var/log/ceph/ceph-client.admin.175131.log: /var/log/ceph/ceph-client.admin.153917.log: /var/log/ceph/ceph-client.admin.145445.log: /var/log/ceph/ceph-client.admin.112642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129103.log 2024-05-04T21:22:40.829 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.242358.log: /var/log/ceph/ceph-client.admin.244412.log: /var/log/ceph/ceph-client.admin.34794.log: /var/log/ceph/ceph-client.admin.322045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.432851.log/var/log/ceph/ceph-client.admin.179771.log: 2024-05-04T21:22:40.829 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.308057.log: /var/log/ceph/ceph-client.admin.159458.log: /var/log/ceph/ceph-client.admin.129103.log: /var/log/ceph/ceph-client.admin.216074.log: /var/log/ceph/ceph-client.admin.35506.log: gzip/var/log/ceph/ceph-client.admin.177178.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.372398.log 2024-05-04T21:22:40.830 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.92248.log: /var/log/ceph/ceph-client.admin.346280.log: /var/log/ceph/ceph-client.admin.112225.log: /var/log/ceph/ceph-client.admin.432851.log: /var/log/ceph/ceph-client.admin.217727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.344164.log 2024-05-04T21:22:40.830 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.181425.log: /var/log/ceph/ceph-client.admin.205903.log: /var/log/ceph/ceph-client.admin.210848.log: 89.1%/var/log/ceph/ceph-client.admin.372398.log: -- replaced with /var/log/ceph/ceph-client.admin.37379.log.gz 2024-05-04T21:22:40.830 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.343342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167661.log 2024-05-04T21:22:40.830 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.105851.log: /var/log/ceph/ceph-client.admin.183456.log: /var/log/ceph/ceph-client.admin.344164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34684.log 2024-05-04T21:22:40.830 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.185269.log: /var/log/ceph/ceph-client.admin.297400.log: /var/log/ceph/ceph-client.admin.154393.log: /var/log/ceph/ceph-client.admin.79487.log: /var/log/ceph/ceph-client.admin.167661.log: /var/log/ceph/ceph-client.admin.222624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.368272.log 2024-05-04T21:22:40.830 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.308048.log: /var/log/ceph/ceph-client.admin.239299.log: /var/log/ceph/ceph-client.admin.35297.log: /var/log/ceph/ceph-client.admin.146664.log: /var/log/ceph/ceph-client.admin.34684.log: /var/log/ceph/ceph-client.admin.33691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150920.log 2024-05-04T21:22:40.830 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.327842.log: /var/log/ceph/ceph-client.admin.359384.log: /var/log/ceph/ceph-client.admin.173099.log: /var/log/ceph/ceph-client.admin.241034.log: /var/log/ceph/ceph-client.admin.450885.log: /var/log/ceph/ceph-client.admin.116759.log: /var/log/ceph/ceph-client.admin.368272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104623.log 2024-05-04T21:22:40.830 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.150920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183447.log 2024-05-04T21:22:40.831 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.343351.log.gz 2024-05-04T21:22:40.831 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.199655.log: /var/log/ceph/ceph-client.admin.120552.log: gzip/var/log/ceph/ceph-client.admin.104623.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.185278.log 2024-05-04T21:22:40.831 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.104207.log: /var/log/ceph/ceph-client.admin.169843.log: /var/log/ceph/ceph-client.admin.197166.log: /var/log/ceph/ceph-client.admin.183447.log: /var/log/ceph/ceph-client.admin.287771.log: /var/log/ceph/ceph-client.admin.225629.log: /var/log/ceph/ceph-client.admin.315400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189392.log 2024-05-04T21:22:40.831 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.127733.log: /var/log/ceph/ceph-client.admin.35454.log: /var/log/ceph/ceph-client.admin.36034.log: /var/log/ceph/ceph-client.admin.125894.log: /var/log/ceph/ceph-client.admin.160733.log: gzip/var/log/ceph/ceph-client.admin.185278.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.163756.log 2024-05-04T21:22:40.831 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.84338.log: /var/log/ceph/ceph-client.admin.169413.log: /var/log/ceph/ceph-client.admin.345873.log: /var/log/ceph/ceph-client.admin.189392.log: /var/log/ceph/ceph-client.admin.229690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.433272.log 2024-05-04T21:22:40.831 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.209610.log: /var/log/ceph/ceph-client.admin.197573.log: /var/log/ceph/ceph-client.admin.328322.log: /var/log/ceph/ceph-client.admin.355222.log: /var/log/ceph/ceph-client.admin.37022.log: /var/log/ceph/ceph-client.admin.36918.log: /var/log/ceph/ceph-client.admin.241960.log: /var/log/ceph/ceph-client.admin.163756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34658.log 2024-05-04T21:22:40.831 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.433272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37223.log 2024-05-04T21:22:40.831 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.386899.log 2024-05-04T21:22:40.832 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34658.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.451697.log.gz 2024-05-04T21:22:40.832 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.37223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36585.log 2024-05-04T21:22:40.832 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.386899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.367866.log 2024-05-04T21:22:40.832 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.215668.log 2024-05-04T21:22:40.832 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.452509.log.gz 2024-05-04T21:22:40.832 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.456576.log 2024-05-04T21:22:40.832 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.367866.log: /var/log/ceph/ceph-client.admin.215668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.242765.log 2024-05-04T21:22:40.832 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.452518.log.gz 2024-05-04T21:22:40.833 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.456576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.232155.log 2024-05-04T21:22:40.833 INFO:teuthology.orchestra.run.smithi052.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.457796.log.gz 2024-05-04T21:22:40.833 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.242765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180605.log 2024-05-04T21:22:40.833 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.459014.log.gz 2024-05-04T21:22:40.833 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.232155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.203380.log 2024-05-04T21:22:40.833 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.180605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.469180.log 2024-05-04T21:22:40.833 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.203380.log: /var/log/ceph/ceph-client.admin.162452.log 2024-05-04T21:22:40.833 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.455358.log 2024-05-04T21:22:40.833 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.469180.log: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.463323.log.gz 2024-05-04T21:22:40.833 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.162452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81718.log 2024-05-04T21:22:40.834 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.455358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.444599.log 2024-05-04T21:22:40.834 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.81718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.322054.log 2024-05-04T21:22:40.834 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.444599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.415174.log 2024-05-04T21:22:40.834 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.322054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.401739.log 2024-05-04T21:22:40.834 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490855.log 2024-05-04T21:22:40.834 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.415174.log: /var/log/ceph/ceph-client.admin.401739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.200062.log 2024-05-04T21:22:40.834 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195102.log 2024-05-04T21:22:40.834 INFO:teuthology.orchestra.run.smithi052.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.490855.log.gz 2024-05-04T21:22:40.835 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.200062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.212120.log 2024-05-04T21:22:40.835 INFO:teuthology.orchestra.run.smithi052.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.488127.log.gz 2024-05-04T21:22:40.835 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.195102.log: 80.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.386079.log.gz -5 2024-05-04T21:22:40.835 INFO:teuthology.orchestra.run.smithi052.stderr: --verbose -- /var/log/ceph/ceph-client.admin.490634.log 2024-05-04T21:22:40.835 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.212120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.467208.log 2024-05-04T21:22:40.835 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.385677.log.gz 2024-05-04T21:22:40.835 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93516.log 2024-05-04T21:22:40.835 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.490634.log.gz 2024-05-04T21:22:40.836 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.467208.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.110957.log 2024-05-04T21:22:40.836 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.93516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144624.log 2024-05-04T21:22:40.836 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.110957.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.219769.log 2024-05-04T21:22:40.836 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.144624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.453732.log 2024-05-04T21:22:40.836 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35532.log 2024-05-04T21:22:40.836 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.219769.log: /var/log/ceph/ceph-client.admin.453732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.306419.log 2024-05-04T21:22:40.836 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.446021.log 2024-05-04T21:22:40.836 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.291469.log 2024-05-04T21:22:40.836 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.306419.log: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.486856.log.gz 2024-05-04T21:22:40.837 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.477849.log 2024-05-04T21:22:40.837 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.446021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.407214.log 2024-05-04T21:22:40.837 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.291469.log: /var/log/ceph/ceph-client.admin.477849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98096.log 2024-05-04T21:22:40.837 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.407214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490660.log 2024-05-04T21:22:40.837 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.98096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.222213.log 2024-05-04T21:22:40.837 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.370732.log 2024-05-04T21:22:40.837 INFO:teuthology.orchestra.run.smithi052.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.490660.log.gz 2024-05-04T21:22:40.837 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.222213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37455.log 2024-05-04T21:22:40.837 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.370732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.301445.log 2024-05-04T21:22:40.838 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36996.log.gz 2024-05-04T21:22:40.838 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.37455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.353625.log 2024-05-04T21:22:40.838 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.301445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110120.log 2024-05-04T21:22:40.838 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.353625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.219362.log 2024-05-04T21:22:40.838 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.110120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.347132.log 2024-05-04T21:22:40.838 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.219362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.420218.log 2024-05-04T21:22:40.838 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5/var/log/ceph/ceph-client.admin.347132.log: --verbose 78.7% -- replaced with /var/log/ceph/ceph-client.admin.426050.log.gz 2024-05-04T21:22:40.838 INFO:teuthology.orchestra.run.smithi052.stderr: -- /var/log/ceph/ceph-client.admin.290639.log 2024-05-04T21:22:40.838 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.420218.log: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.426520.log.gz 2024-05-04T21:22:40.839 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.197970.log 2024-05-04T21:22:40.839 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152188.log 90.3% 2024-05-04T21:22:40.839 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.425211.log.gz 2024-05-04T21:22:40.839 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.290639.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.426059.log.gz 2024-05-04T21:22:40.839 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.197970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.365389.log 2024-05-04T21:22:40.839 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.152188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.230097.log 2024-05-04T21:22:40.839 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.365389.log: gzip 78.9% -- replaced with /var/log/ceph/ceph-client.admin.478268.log.gz 2024-05-04T21:22:40.839 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.477420.log 2024-05-04T21:22:40.839 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.230097.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22714.log 2024-05-04T21:22:40.840 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.477420.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.488609.log.gz 2024-05-04T21:22:40.840 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.460253.log 2024-05-04T21:22:40.840 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph.tmp-client.admin.22714.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.470643.log.gz 2024-05-04T21:22:40.840 INFO:teuthology.orchestra.run.smithi052.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22714.log.gz 2024-05-04T21:22:40.840 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.437161.log 2024-05-04T21:22:40.840 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.460253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118050.log 2024-05-04T21:22:40.840 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.437161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.328829.log 2024-05-04T21:22:40.840 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.118050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.220176.log 2024-05-04T21:22:40.840 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.397802.log.gz 2024-05-04T21:22:40.841 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.328829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179762.log 2024-05-04T21:22:40.841 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.220176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151753.log 2024-05-04T21:22:40.841 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.179762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179331.log 2024-05-04T21:22:40.841 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.363306.log.gz 2024-05-04T21:22:40.841 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.151753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.464587.log 2024-05-04T21:22:40.841 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.179331.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.369497.log.gzgzip 2024-05-04T21:22:40.841 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.466353.log 2024-05-04T21:22:40.841 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.464587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.285311.log 2024-05-04T21:22:40.841 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.213141.log 2024-05-04T21:22:40.842 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.466353.log: /var/log/ceph/ceph-client.admin.285311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35589.log 2024-05-04T21:22:40.842 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.380906.log.gz 2024-05-04T21:22:40.842 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.213141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.460680.log 2024-05-04T21:22:40.842 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.374454.log 2024-05-04T21:22:40.842 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.460680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162887.log 2024-05-04T21:22:40.842 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.374454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83053.log 2024-05-04T21:22:40.842 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.162887.log: /var/log/ceph/ceph-client.admin.83053.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-04T21:22:40.842 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.395318.log.gz 2024-05-04T21:22:40.842 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33495.log 2024-05-04T21:22:40.843 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118886.log 2024-05-04T21:22:40.843 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154796.log 2024-05-04T21:22:40.843 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.118886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.464578.log 2024-05-04T21:22:40.843 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.154796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.283273.log 2024-05-04T21:22:40.843 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.464578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.452915.log 2024-05-04T21:22:40.843 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.398274.log.gz 2024-05-04T21:22:40.843 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.283273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.419622.log 2024-05-04T21:22:40.843 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.452915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106706.log 2024-05-04T21:22:40.844 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.419622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491119.log 2024-05-04T21:22:40.844 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.106706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.199646.log 2024-05-04T21:22:40.844 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.491119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129094.log 2024-05-04T21:22:40.844 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.199646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181831.log 2024-05-04T21:22:40.844 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.491119.log.gz 2024-05-04T21:22:40.844 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5/var/log/ceph/ceph-client.admin.129094.log: --verbose -- /var/log/ceph/ceph-client.admin.439273.log 2024-05-04T21:22:40.844 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.181831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.456170.log 2024-05-04T21:22:40.844 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.401739.log.gz 2024-05-04T21:22:40.844 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.439273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.226432.log 2024-05-04T21:22:40.844 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.456170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.480423.log 2024-05-04T21:22:40.845 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.226432.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.188161.log.gz 2024-05-04T21:22:40.845 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192473.log 2024-05-04T21:22:40.845 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.480423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169852.log 2024-05-04T21:22:40.845 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.192473.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.188170.log.gz 2024-05-04T21:22:40.845 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.296995.log 2024-05-04T21:22:40.845 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.169852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117240.log 2024-05-04T21:22:40.845 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.296995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.410175.log 2024-05-04T21:22:40.845 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.117240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33026.log 2024-05-04T21:22:40.846 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.410175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178015.log 2024-05-04T21:22:40.846 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33026.log: gzip -5 90.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.176364.log.gz 2024-05-04T21:22:40.846 INFO:teuthology.orchestra.run.smithi052.stderr: /var/log/ceph/ceph-client.admin.184863.log 2024-05-04T21:22:40.846 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.178015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.394903.log 2024-05-04T21:22:40.846 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.184863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165052.log 2024-05-04T21:22:40.846 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.394903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181011.log 2024-05-04T21:22:40.846 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.165052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.466802.log 2024-05-04T21:22:40.846 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.181011.log: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.375261.log.gzgzip 2024-05-04T21:22:40.846 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.346716.log 2024-05-04T21:22:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.466802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175140.log 2024-05-04T21:22:40.847 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.375270.log.gz 2024-05-04T21:22:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.346716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.373222.log 2024-05-04T21:22:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.175140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.447596.log 2024-05-04T21:22:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.373222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.377334.log 2024-05-04T21:22:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.447596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.354321.log 2024-05-04T21:22:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.377334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.211245.log 2024-05-04T21:22:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.354321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151355.log 2024-05-04T21:22:40.848 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.211245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.224719.log 2024-05-04T21:22:40.848 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.151355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.201692.log 2024-05-04T21:22:40.848 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.224719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.200868.log 2024-05-04T21:22:40.848 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.201692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.473586.log 2024-05-04T21:22:40.848 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.200868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150911.log 2024-05-04T21:22:40.848 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.473586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33483.log 2024-05-04T21:22:40.848 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.150911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.397375.log 2024-05-04T21:22:40.848 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.329778.log 2024-05-04T21:22:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.397375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183041.log 2024-05-04T21:22:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.329778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.384628.log 2024-05-04T21:22:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.183041.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.459827.log.gz 2024-05-04T21:22:40.849 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.195102.log.gz 2024-05-04T21:22:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118476.log 2024-05-04T21:22:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.384628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167227.log 2024-05-04T21:22:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.118476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.395715.log 2024-05-04T21:22:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.167227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.481261.log 2024-05-04T21:22:40.850 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.395715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117231.log 2024-05-04T21:22:40.850 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.481261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490452.log 2024-05-04T21:22:40.850 INFO:teuthology.orchestra.run.smithi052.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.418657.log.gz 2024-05-04T21:22:40.850 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.327833.log 2024-05-04T21:22:40.850 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.418195.log.gz 2024-05-04T21:22:40.850 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119731.log 2024-05-04T21:22:40.850 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.490452.log.gz/var/log/ceph/ceph-client.admin.327833.log: 2024-05-04T21:22:40.850 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194298.log 2024-05-04T21:22:40.851 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.420218.log.gz 2024-05-04T21:22:40.851 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.117231.log: /var/log/ceph/ceph-client.admin.119731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.241518.log 2024-05-04T21:22:40.851 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.194298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.218540.log 2024-05-04T21:22:40.851 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.241518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194696.log 2024-05-04T21:22:40.851 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.218540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119740.log 2024-05-04T21:22:40.851 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.194696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.234250.log 2024-05-04T21:22:40.851 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.119740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.369893.log 2024-05-04T21:22:40.851 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.234250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101360.log 2024-05-04T21:22:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.369893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93090.log 2024-05-04T21:22:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.101360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155276.log 2024-05-04T21:22:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.93090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182644.log 2024-05-04T21:22:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.155276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.424351.log 2024-05-04T21:22:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.182644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.299417.log 2024-05-04T21:22:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.424351.log: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.152197.log.gz 2024-05-04T21:22:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110538.log 2024-05-04T21:22:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.299417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.281240.log 2024-05-04T21:22:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.110538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119318.log 2024-05-04T21:22:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.281240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34098.log 2024-05-04T21:22:40.853 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36533.log.gz 2024-05-04T21:22:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.119318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34046.log 2024-05-04T21:22:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27801.log 2024-05-04T21:22:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162046.log 2024-05-04T21:22:40.853 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36812.log.gz 2024-05-04T21:22:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.27801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107572.log 2024-05-04T21:22:40.854 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.162046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36091.log 2024-05-04T21:22:40.854 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.107572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.436736.log 2024-05-04T21:22:40.854 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.452103.log 2024-05-04T21:22:40.854 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.436736.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34254.log.gz 2024-05-04T21:22:40.854 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144222.log 2024-05-04T21:22:40.854 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.452103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105440.log 2024-05-04T21:22:40.854 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.144222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.434075.log 2024-05-04T21:22:40.854 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.105440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.461096.log 2024-05-04T21:22:40.854 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.434075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.323857.log 2024-05-04T21:22:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.461096.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32848.log.gz 2024-05-04T21:22:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.377740.log 2024-05-04T21:22:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.323857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.428304.log 2024-05-04T21:22:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.377740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36247.log 2024-05-04T21:22:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.428304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126383.log 2024-05-04T21:22:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81311.log 2024-05-04T21:22:40.855 INFO:teuthology.orchestra.run.smithi052.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.458202.log.gz 2024-05-04T21:22:40.855 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.460253.log.gz 2024-05-04T21:22:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.126383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.430001.log 2024-05-04T21:22:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.81311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.382539.log 2024-05-04T21:22:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.430001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150066.log 2024-05-04T21:22:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.382539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.288586.log 2024-05-04T21:22:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.150066.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.85377.log 2024-05-04T21:22:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.288586.log: /var/log/ceph/ceph-client.admin.85377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166357.log 2024-05-04T21:22:40.856 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.408487.log.gz 2024-05-04T21:22:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.417788.log 2024-05-04T21:22:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.166357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.303097.log 2024-05-04T21:22:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.417788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.414739.log 2024-05-04T21:22:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.303097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.465469.log 2024-05-04T21:22:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.414739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193883.log 2024-05-04T21:22:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.465469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159449.log 2024-05-04T21:22:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.193883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491327.log 2024-05-04T21:22:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.159449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.428295.log 2024-05-04T21:22:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.491327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.352371.log 2024-05-04T21:22:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.428295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.407621.log 88.6% 2024-05-04T21:22:40.857 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.491327.log.gz 2024-05-04T21:22:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.352371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95584.log 2024-05-04T21:22:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.407621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.234690.log 2024-05-04T21:22:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.95584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.401332.log 2024-05-04T21:22:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.234690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33132.log 2024-05-04T21:22:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.401332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34176.log 2024-05-04T21:22:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.408072.log 2024-05-04T21:22:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33132.log: /var/log/ceph/ceph-client.admin.34176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.218133.log 2024-05-04T21:22:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.408072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.314579.log 2024-05-04T21:22:40.859 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.218133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33690.log 2024-05-04T21:22:40.859 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.314579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.402141.log 2024-05-04T21:22:40.859 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37191.log 2024-05-04T21:22:40.859 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.402141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.485991.log 2024-05-04T21:22:40.859 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.37191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.484752.log 2024-05-04T21:22:40.859 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.485991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.311388.log 2024-05-04T21:22:40.859 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.484752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.315798.log 2024-05-04T21:22:40.859 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.311388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.320015.log 2024-05-04T21:22:40.860 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.315798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.311785.log 2024-05-04T21:22:40.860 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.320015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.290212.log 2024-05-04T21:22:40.860 INFO:teuthology.orchestra.run.smithi052.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.288173.log.gz 2024-05-04T21:22:40.860 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.204210.log 2024-05-04T21:22:40.860 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.290212.log: /var/log/ceph/ceph-client.admin.311785.log: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.288586.log.gz 2024-05-04T21:22:40.860 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.358218.log 2024-05-04T21:22:40.860 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.287771.log.gz 2024-05-04T21:22:40.860 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.204210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126392.log 2024-05-04T21:22:40.861 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.358218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.300642.log 2024-05-04T21:22:40.861 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.126392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.298211.log 2024-05-04T21:22:40.861 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.300642.log: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.416397.log.gz 2024-05-04T21:22:40.861 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155267.log 2024-05-04T21:22:40.861 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.298211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150476.log 2024-05-04T21:22:40.861 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.155267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.347602.log 2024-05-04T21:22:40.861 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.150476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.370321.log 2024-05-04T21:22:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.347602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.241025.log 2024-05-04T21:22:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.370321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.394051.log 2024-05-04T21:22:40.862 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.453741.log.gz 2024-05-04T21:22:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.241025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190711.log 2024-05-04T21:22:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.394051.log: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.462007.log.gz 2024-05-04T21:22:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.212549.log 2024-05-04T21:22:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.190711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183876.log 2024-05-04T21:22:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.212549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.286951.log 2024-05-04T21:22:40.863 INFO:teuthology.orchestra.run.smithi052.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.452103.log.gz 2024-05-04T21:22:40.863 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.183876.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.413775.log 2024-05-04T21:22:40.863 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.451706.log.gz 2024-05-04T21:22:40.863 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.286951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490081.log 2024-05-04T21:22:40.863 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.413775.log: gzip 80.7% -- replaced with /var/log/ceph/ceph-client.admin.457390.log.gz 2024-05-04T21:22:40.863 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.93931.log 2024-05-04T21:22:40.863 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107117.log 2024-05-04T21:22:40.863 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.93931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.319600.log 2024-05-04T21:22:40.864 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.490081.log.gz 2024-05-04T21:22:40.864 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.107117.log: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.394042.log.gz 2024-05-04T21:22:40.864 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.296181.log 2024-05-04T21:22:40.864 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.319600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.329302.log 2024-05-04T21:22:40.864 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.296181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.454138.log 2024-05-04T21:22:40.864 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.329302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166366.log 2024-05-04T21:22:40.864 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.454138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.467628.log 2024-05-04T21:22:40.864 INFO:teuthology.orchestra.run.smithi052.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.439264.log.gz 2024-05-04T21:22:40.865 INFO:teuthology.orchestra.run.smithi052.stderr: 85.4%/var/log/ceph/ceph-client.admin.166366.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.438858.log.gz -5 --verbose 2024-05-04T21:22:40.865 INFO:teuthology.orchestra.run.smithi052.stderr: -- /var/log/ceph/ceph-client.admin.36143.log 2024-05-04T21:22:40.865 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.467628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.458617.log 2024-05-04T21:22:40.865 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.384628.log.gz 2024-05-04T21:22:40.865 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37048.log 2024-05-04T21:22:40.865 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.458617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.460671.log 2024-05-04T21:22:40.865 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.37048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.464135.log 2024-05-04T21:22:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.460671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.386088.log 2024-05-04T21:22:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.464135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.451300.log 2024-05-04T21:22:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.386088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35797.log 2024-05-04T21:22:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.451300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.475703.log 2024-05-04T21:22:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35184.log 2024-05-04T21:22:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.475703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.398737.log 2024-05-04T21:22:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.365801.log 2024-05-04T21:22:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.398737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.487313.log 2024-05-04T21:22:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.365801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.321639.log 2024-05-04T21:22:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.487313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.376074.log 2024-05-04T21:22:40.867 INFO:teuthology.orchestra.run.smithi052.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.441334.log.gz 2024-05-04T21:22:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.321639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147905.log 2024-05-04T21:22:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.376074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192036.log 2024-05-04T21:22:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.313639.log 2024-05-04T21:22:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.147905.log: /var/log/ceph/ceph-client.admin.192036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82125.log 2024-05-04T21:22:40.867 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.456179.log.gz 2024-05-04T21:22:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.356498.log 83.1% 2024-05-04T21:22:40.868 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.181011.log.gz 2024-05-04T21:22:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.313639.log: /var/log/ceph/ceph-client.admin.82125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35323.log 2024-05-04T21:22:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.356498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.374445.log 2024-05-04T21:22:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35323.log: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.395715.log.gz 2024-05-04T21:22:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.452924.log 2024-05-04T21:22:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.374445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181020.log 2024-05-04T21:22:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.452924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.434519.log 2024-05-04T21:22:40.868 INFO:teuthology.orchestra.run.smithi052.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.473577.log.gz 2024-05-04T21:22:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183050.log 2024-05-04T21:22:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.181020.log: /var/log/ceph/ceph-client.admin.434519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.369488.log 2024-05-04T21:22:40.869 INFO:teuthology.orchestra.run.smithi052.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.397375.log.gz 2024-05-04T21:22:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.183050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.475264.log 2024-05-04T21:22:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.369488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.233816.log 2024-05-04T21:22:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.475264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80444.log 2024-05-04T21:22:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.233816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.221400.log 2024-05-04T21:22:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.80444.log: -5 --verbose 81.5% -- replaced with /var/log/ceph/ceph-client.admin.410166.log.gz 2024-05-04T21:22:40.869 INFO:teuthology.orchestra.run.smithi052.stderr: -- /var/log/ceph/ceph-client.admin.283688.log/var/log/ceph/ceph-client.admin.221400.log: 2024-05-04T21:22:40.870 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose 90.3% -- 90.2% -- replaced with /var/log/ceph/ceph-client.admin.410175.log.gz /var/log/ceph/ceph-client.admin.412831.log 2024-05-04T21:22:40.870 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.409705.log.gz 2024-05-04T21:22:40.870 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-04T21:22:40.870 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.283688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148339.log 2024-05-04T21:22:40.870 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.412831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36970.log 2024-05-04T21:22:40.870 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.148339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.415999.log 2024-05-04T21:22:40.870 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.206344.log 2024-05-04T21:22:40.870 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.365389.log.gz 2024-05-04T21:22:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.415999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82594.log 2024-05-04T21:22:40.871 INFO:teuthology.orchestra.run.smithi052.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.367045.log.gz 2024-05-04T21:22:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.206344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180614.log 2024-05-04T21:22:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.82594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189401.log 2024-05-04T21:22:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.180614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35264.log 2024-05-04T21:22:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.189401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.294929.log 2024-05-04T21:22:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.347611.log 2024-05-04T21:22:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.294929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.326875.log 2024-05-04T21:22:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.347611.log: gzip -5 --verbose 80.0% -- replaced with /var/log/ceph/ceph-client.admin.327833.log.gz 2024-05-04T21:22:40.872 INFO:teuthology.orchestra.run.smithi052.stderr: -- /var/log/ceph/ceph-client.admin.201683.log 2024-05-04T21:22:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.326875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98105.log 2024-05-04T21:22:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.201683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.218951.log 2024-05-04T21:22:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.98105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149234.log 2024-05-04T21:22:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.218951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.475273.log 2024-05-04T21:22:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.149234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150057.log 2024-05-04T21:22:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.475273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.368679.log 2024-05-04T21:22:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.150057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.441771.log 2024-05-04T21:22:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.365792.log 2024-05-04T21:22:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.368679.log: /var/log/ceph/ceph-client.admin.441771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.240122.log 2024-05-04T21:22:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.365792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.242367.log 2024-05-04T21:22:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.240122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145856.log 2024-05-04T21:22:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.242367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.346725.log 2024-05-04T21:22:40.873 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.306011.log.gz 2024-05-04T21:22:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.145856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.214794.log 2024-05-04T21:22:40.873 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.306419.log.gz 2024-05-04T21:22:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.346725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.415585.log 2024-05-04T21:22:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.214794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.421910.log 2024-05-04T21:22:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.415585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.230501.log 2024-05-04T21:22:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.421910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.351964.log 2024-05-04T21:22:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.230501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.354330.log 2024-05-04T21:22:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.351964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178885.log 2024-05-04T21:22:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.354330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.244002.log 2024-05-04T21:22:40.874 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.220176.log.gz 2024-05-04T21:22:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.178885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79989.log 2024-05-04T21:22:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.244002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86255.log 2024-05-04T21:22:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.79989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35667.log 2024-05-04T21:22:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.86255.log: 79.9% -- replaced with /var/log/ceph/ceph-client.admin.183041.log.gz 2024-05-04T21:22:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.310973.log 2024-05-04T21:22:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35667.log: gzip 90.4% -- replaced with /var/log/ceph/ceph-client.admin.182644.log.gz 2024-05-04T21:22:40.875 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35745.log 2024-05-04T21:22:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.310973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.242774.log 2024-05-04T21:22:40.876 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191590.log 2024-05-04T21:22:40.876 INFO:teuthology.orchestra.run.smithi052.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.347602.log.gz 2024-05-04T21:22:40.876 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.242774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.358630.log 2024-05-04T21:22:40.876 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.191590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.314588.log 2024-05-04T21:22:40.876 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.358630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.402561.log 2024-05-04T21:22:40.876 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.314588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.301039.log 2024-05-04T21:22:40.877 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.402561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491456.log 2024-05-04T21:22:40.877 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.301039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.208750.log 2024-05-04T21:22:40.877 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.491456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.300236.log 2024-05-04T21:22:40.877 INFO:teuthology.orchestra.run.smithi052.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.491456.log.gz 2024-05-04T21:22:40.877 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.208750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151762.log 2024-05-04T21:22:40.877 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.300236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.307231.log 2024-05-04T21:22:40.877 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.151762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.350739.log 2024-05-04T21:22:40.877 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.307231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.411869.log 2024-05-04T21:22:40.878 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.350739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.205912.log 2024-05-04T21:22:40.878 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.411869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.417309.log 2024-05-04T21:22:40.878 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.205912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32882.log 2024-05-04T21:22:40.878 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.417309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188995.log 2024-05-04T21:22:40.878 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.304367.log 2024-05-04T21:22:40.878 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.32882.log: /var/log/ceph/ceph-client.admin.188995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.403364.log 2024-05-04T21:22:40.878 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.304367.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.119740.log.gz 2024-05-04T21:22:40.878 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145030.log 2024-05-04T21:22:40.878 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.403364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.405934.log 2024-05-04T21:22:40.879 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.145030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159868.log 2024-05-04T21:22:40.879 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.405934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.483005.log 2024-05-04T21:22:40.879 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119327.log 2024-05-04T21:22:40.879 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.483005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153467.log 2024-05-04T21:22:40.879 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.377334.log.gz 2024-05-04T21:22:40.879 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.119327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.202530.log 2024-05-04T21:22:40.879 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.159868.log: /var/log/ceph/ceph-client.admin.153467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.421031.log 2024-05-04T21:22:40.879 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.202530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33501.log 2024-05-04T21:22:40.880 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.421031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.367054.log 2024-05-04T21:22:40.880 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.455367.log 2024-05-04T21:22:40.880 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.367054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.407223.log 2024-05-04T21:22:40.880 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.380499.log.gz 2024-05-04T21:22:40.880 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.455367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102181.log 2024-05-04T21:22:40.880 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.407223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.285731.log 2024-05-04T21:22:40.880 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.102181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.199240.log 2024-05-04T21:22:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.285731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.377731.log 2024-05-04T21:22:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.199240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35851.log 2024-05-04T21:22:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.377731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.196354.log 2024-05-04T21:22:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111810.log 2024-05-04T21:22:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.196345.log 2024-05-04T21:22:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.196354.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.430401.log.gz 2024-05-04T21:22:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.111810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.218531.log 2024-05-04T21:22:40.881 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.430001.log.gz 2024-05-04T21:22:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.196345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.412840.log 2024-05-04T21:22:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.218531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.417318.log 2024-05-04T21:22:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.412840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.402967.log 2024-05-04T21:22:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.417318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.478697.log 2024-05-04T21:22:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.402967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.403771.log 2024-05-04T21:22:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.213585.log 2024-05-04T21:22:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.478697.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.432851.log.gz 2024-05-04T21:22:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.403771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.450666.log 2024-05-04T21:22:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.329311.log 2024-05-04T21:22:40.883 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.213585.log: /var/log/ceph/ceph-client.admin.450666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.461605.log 2024-05-04T21:22:40.883 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.434510.log.gz 2024-05-04T21:22:40.883 INFO:teuthology.orchestra.run.smithi052.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.434075.log.gz 2024-05-04T21:22:40.883 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.329311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.210839.log 2024-05-04T21:22:40.883 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.433677.log.gz 2024-05-04T21:22:40.883 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.461605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193436.log 2024-05-04T21:22:40.884 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.210839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.356961.log 2024-05-04T21:22:40.884 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.193436.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.437161.log.gz 2024-05-04T21:22:40.884 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33704.log 2024-05-04T21:22:40.884 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.356961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.217316.log 2024-05-04T21:22:40.884 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.320828.log 2024-05-04T21:22:40.884 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36721.log 2024-05-04T21:22:40.884 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.217316.log: /var/log/ceph/ceph-client.admin.320828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.284911.log 2024-05-04T21:22:40.884 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.452924.log.gz 2024-05-04T21:22:40.885 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175539.log 2024-05-04T21:22:40.885 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36721.log: /var/log/ceph/ceph-client.admin.284911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.291050.log 2024-05-04T21:22:40.885 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.175539.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-04T21:22:40.885 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.438055.log 2024-05-04T21:22:40.885 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.291050.log: /var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.303559.log 2024-05-04T21:22:40.885 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.286142.log.gz 2024-05-04T21:22:40.885 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.438055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33186.log 2024-05-04T21:22:40.885 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.467628.log.gz 2024-05-04T21:22:40.885 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.303559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.298607.log 2024-05-04T21:22:40.886 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.432448.log 2024-05-04T21:22:40.886 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.298607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.459023.log 2024-05-04T21:22:40.886 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.432448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118467.log 2024-05-04T21:22:40.886 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.386088.log.gz 2024-05-04T21:22:40.886 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.459023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.440485.log 2024-05-04T21:22:40.886 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.118467.log: /var/log/ceph/ceph-client.admin.440485.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.181020.log.gz 2024-05-04T21:22:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195509.log 2024-05-04T21:22:40.887 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.370732.log.gz 2024-05-04T21:22:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146673.log 2024-05-04T21:22:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.195509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179322.log 2024-05-04T21:22:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.288595.log 2024-05-04T21:22:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.146673.log: /var/log/ceph/ceph-client.admin.179322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.351145.log 2024-05-04T21:22:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.288595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.416846.log 2024-05-04T21:22:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.351145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.358621.log 2024-05-04T21:22:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.416846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113063.log 2024-05-04T21:22:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.358621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33000.log 2024-05-04T21:22:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.113063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178024.log 2024-05-04T21:22:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33000.log: /var/log/ceph/ceph-client.admin.178024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.462443.log 2024-05-04T21:22:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.422310.log 2024-05-04T21:22:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.462443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.474420.log 2024-05-04T21:22:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.422310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109717.log 2024-05-04T21:22:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.474420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106253.log 2024-05-04T21:22:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.109717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.230088.log 2024-05-04T21:22:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.106253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.225620.log 2024-05-04T21:22:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.230088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35771.log 2024-05-04T21:22:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.225620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.330248.log 2024-05-04T21:22:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117638.log 2024-05-04T21:22:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.330248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148774.log 2024-05-04T21:22:40.890 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.117638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.470223.log 2024-05-04T21:22:40.890 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.148774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.231313.log 2024-05-04T21:22:40.890 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.470223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.427332.log 2024-05-04T21:22:40.890 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.231313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.385257.log 2024-05-04T21:22:40.890 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.427332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.439672.log 2024-05-04T21:22:40.890 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.385257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.384637.log 2024-05-04T21:22:40.890 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.217316.log.gz 2024-05-04T21:22:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.439672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.351136.log 2024-05-04T21:22:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.384637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34280.log 2024-05-04T21:22:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.351136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.438046.log 2024-05-04T21:22:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.355628.log 2024-05-04T21:22:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34280.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.219760.log.gz 2024-05-04T21:22:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.438046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.281650.log 2024-05-04T21:22:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.323387.log 2024-05-04T21:22:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.355628.log: /var/log/ceph/ceph-client.admin.281650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.216485.log 2024-05-04T21:22:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.382128.log 2024-05-04T21:22:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.323387.log: /var/log/ceph/ceph-client.admin.216485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.228877.log 2024-05-04T21:22:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.382128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.419613.log 2024-05-04T21:22:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.228877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.430807.log 2024-05-04T21:22:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.419613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.364974.log 2024-05-04T21:22:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.430807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96796.log 2024-05-04T21:22:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.364974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.460262.log 2024-05-04T21:22:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.96796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.412368.log 2024-05-04T21:22:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.460262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.344173.log 2024-05-04T21:22:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.412368.log: 81.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.478697.log.gz -5 2024-05-04T21:22:40.893 INFO:teuthology.orchestra.run.smithi052.stderr: --verbose -- /var/log/ceph/ceph-client.admin.476982.log 2024-05-04T21:22:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.344173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.197979.log 2024-05-04T21:22:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.476982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.383376.log 2024-05-04T21:22:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.197979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33694.log 2024-05-04T21:22:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.383376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.479565.log 2024-05-04T21:22:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.326039.log 2024-05-04T21:22:40.894 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33694.log: /var/log/ceph/ceph-client.admin.479565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.485982.log 2024-05-04T21:22:40.894 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.326039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.369086.log 2024-05-04T21:22:40.894 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.485982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.444590.log 2024-05-04T21:22:40.894 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.369086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490783.log 2024-05-04T21:22:40.894 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.444590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34020.log 2024-05-04T21:22:40.894 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95178.log 2024-05-04T21:22:40.894 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.290630.log 2024-05-04T21:22:40.894 INFO:teuthology.orchestra.run.smithi052.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.490783.log.gz 2024-05-04T21:22:40.894 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.95178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84971.log 2024-05-04T21:22:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.305588.log 2024-05-04T21:22:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.290630.log: /var/log/ceph/ceph-client.admin.84971.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.487313.log.gz 2024-05-04T21:22:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.478706.log 2024-05-04T21:22:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.305588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172287.log 2024-05-04T21:22:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.478706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128139.log 2024-05-04T21:22:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108899.log 2024-05-04T21:22:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.172287.log: /var/log/ceph/ceph-client.admin.128139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195111.log 2024-05-04T21:22:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.108899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.222633.log 2024-05-04T21:22:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.195111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176355.log 2024-05-04T21:22:40.896 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.222633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.401323.log 2024-05-04T21:22:40.896 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.176355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156437.log 2024-05-04T21:22:40.896 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.401323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.282455.log 2024-05-04T21:22:40.896 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.156437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.306816.log 2024-05-04T21:22:40.896 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.282455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152631.log 2024-05-04T21:22:40.896 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.306816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.472756.log 2024-05-04T21:22:40.896 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.152631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34513.log 2024-05-04T21:22:40.896 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.472756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.342468.log 2024-05-04T21:22:40.897 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.214388.log 2024-05-04T21:22:40.897 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.342468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177589.log 2024-05-04T21:22:40.897 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.214388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.348253.log 2024-05-04T21:22:40.897 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.177589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.301860.log 2024-05-04T21:22:40.897 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.348253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.352776.log 2024-05-04T21:22:40.897 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.301860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.467619.log 2024-05-04T21:22:40.897 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.352776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33448.log 2024-05-04T21:22:40.897 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.467619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85848.log 2024-05-04T21:22:40.897 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.376480.log 2024-05-04T21:22:40.898 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.298607.log.gz 2024-05-04T21:22:40.898 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.85848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.471471.log 2024-05-04T21:22:40.898 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.376480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185684.log 2024-05-04T21:22:40.898 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.471471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36195.log 2024-05-04T21:22:40.898 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.185684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34716.log 2024-05-04T21:22:40.898 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146253.log 2024-05-04T21:22:40.898 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.452112.log 2024-05-04T21:22:40.901 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.146253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33160.log 2024-05-04T21:22:40.901 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.452112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.400885.log 2024-05-04T21:22:40.902 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33160.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.300642.log.gz 2024-05-04T21:22:40.902 INFO:teuthology.orchestra.run.smithi052.stderr: 82.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.196345.log.gz 2024-05-04T21:22:40.902 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.94762.log 2024-05-04T21:22:40.902 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.400885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99733.log 2024-05-04T21:22:40.902 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.94762.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.196354.log.gz 2024-05-04T21:22:40.902 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128130.log 2024-05-04T21:22:40.902 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.99733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159859.log 2024-05-04T21:22:40.902 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.128130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.297391.log 2024-05-04T21:22:40.903 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.159859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.292400.log 2024-05-04T21:22:40.903 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.297391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.213991.log 2024-05-04T21:22:40.903 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.292400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.294523.log 2024-05-04T21:22:40.903 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.213991.log: /var/log/ceph/ceph-client.admin.294523.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.292815.log.gz 2024-05-04T21:22:40.903 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.481690.log 2024-05-04T21:22:40.903 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.400479.log 2024-05-04T21:22:40.903 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.481690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.311379.log 2024-05-04T21:22:40.903 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.400479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.381717.log 2024-05-04T21:22:40.904 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.213576.log 2024-05-04T21:22:40.904 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.311379.log: /var/log/ceph/ceph-client.admin.381717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.227244.log 2024-05-04T21:22:40.904 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.223859.log 2024-05-04T21:22:40.904 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.213576.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.294929.log.gz 2024-05-04T21:22:40.904 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.227244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.352785.log 2024-05-04T21:22:40.904 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.223859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100546.log 2024-05-04T21:22:40.904 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.352785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.472348.log 2024-05-04T21:22:40.904 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.100546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.449047.log 2024-05-04T21:22:40.905 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.472348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.413303.log 2024-05-04T21:22:40.905 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.449047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.455764.log 2024-05-04T21:22:40.905 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.413303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.415165.log 2024-05-04T21:22:40.905 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.455764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174732.log 2024-05-04T21:22:40.905 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.415165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162037.log 2024-05-04T21:22:40.905 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.174732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.343753.log 2024-05-04T21:22:40.905 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.162037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.404186.log 2024-05-04T21:22:40.906 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.343753.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-04T21:22:40.906 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.404186.log: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.290212.log.gz 2024-05-04T21:22:40.906 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.305181.log 2024-05-04T21:22:40.906 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96386.log 2024-05-04T21:22:40.906 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.305181.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.165496.log 2024-05-04T21:22:40.906 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.96386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.208741.log 2024-05-04T21:22:40.906 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.165496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182228.log 2024-05-04T21:22:40.906 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.208741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.368263.log 2024-05-04T21:22:40.907 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.182228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.366648.log 2024-05-04T21:22:40.907 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.368263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33396.log 2024-05-04T21:22:40.907 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.366648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92664.log 2024-05-04T21:22:40.907 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.219353.log 2024-05-04T21:22:40.907 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.92664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490400.log 2024-05-04T21:22:40.907 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.219353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.382965.log 2024-05-04T21:22:40.907 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.421022.log 2024-05-04T21:22:40.908 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.490400.log.gz 2024-05-04T21:22:40.908 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.382965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187755.log 2024-05-04T21:22:40.908 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.421022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.471462.log 2024-05-04T21:22:40.908 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.187755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195924.log 2024-05-04T21:22:40.908 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.471462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.376919.log 2024-05-04T21:22:40.908 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.195924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.450894.log 2024-05-04T21:22:40.908 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.95178.log.gz 2024-05-04T21:22:40.908 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.376919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97697.log 2024-05-04T21:22:40.909 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.450894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122737.log 2024-05-04T21:22:40.909 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.97697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147896.log 2024-05-04T21:22:40.909 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.122737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.465916.log 2024-05-04T21:22:40.909 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.147896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.352362.log 2024-05-04T21:22:40.909 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.465916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84558.log 2024-05-04T21:22:40.909 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423124.log 2024-05-04T21:22:40.909 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.352362.log: /var/log/ceph/ceph-client.admin.84558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.357368.log 2024-05-04T21:22:40.909 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.423124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85386.log 2024-05-04T21:22:40.910 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.357368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34820.log 2024-05-04T21:22:40.910 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.85386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.381708.log 2024-05-04T21:22:40.910 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.302681.log 2024-05-04T21:22:40.910 INFO:teuthology.orchestra.run.smithi052.stderr: 77.6% -- replaced with /var/log/ceph/ceph-client.admin.157646.log.gz 2024-05-04T21:22:40.910 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97252.log 2024-05-04T21:22:40.910 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.381708.log: /var/log/ceph/ceph-client.admin.302681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.240131.log 2024-05-04T21:22:40.910 INFO:teuthology.orchestra.run.smithi052.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.193436.log.gz 2024-05-04T21:22:40.911 INFO:teuthology.orchestra.run.smithi052.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.193883.log.gz 2024-05-04T21:22:40.911 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.345476.log 2024-05-04T21:22:40.911 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.97252.log: /var/log/ceph/ceph-client.admin.240131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.397793.log 2024-05-04T21:22:40.911 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.345476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159023.log 2024-05-04T21:22:40.911 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.100546.log.gz 2024-05-04T21:22:40.911 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.397793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182237.log 2024-05-04T21:22:40.911 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.159023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.405115.log 2024-05-04T21:22:40.911 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.100555.log.gz 2024-05-04T21:22:40.912 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.182237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35028.log 2024-05-04T21:22:40.912 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.405115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117647.log 2024-05-04T21:22:40.912 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.199646.log.gz 2024-05-04T21:22:40.912 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153028.log 2024-05-04T21:22:40.912 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.117647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35401.log 2024-05-04T21:22:40.912 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.153028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.435778.log 2024-05-04T21:22:40.912 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491430.log 2024-05-04T21:22:40.912 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.435778.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.291041.log 2024-05-04T21:22:40.912 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.491430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.472339.log 2024-05-04T21:22:40.913 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.491430.log.gz 2024-05-04T21:22:40.913 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.291041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.394506.log 2024-05-04T21:22:40.913 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.472339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.225214.log 2024-05-04T21:22:40.913 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.394506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168105.log 2024-05-04T21:22:40.913 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.225214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27683.log 2024-05-04T21:22:40.913 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.226432.log.gz 2024-05-04T21:22:40.913 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.168105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169404.log 2024-05-04T21:22:40.913 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.27683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163312.log 2024-05-04T21:22:40.913 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.169404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.302270.log 2024-05-04T21:22:40.914 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.163312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.288182.log 2024-05-04T21:22:40.914 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.302270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.318367.log 2024-05-04T21:22:40.914 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.288182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110129.log 2024-05-04T21:22:40.914 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.318367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.211660.log 2024-05-04T21:22:40.914 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.110129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.289816.log 2024-05-04T21:22:40.914 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.211660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490885.log 2024-05-04T21:22:40.914 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.289816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120561.log 2024-05-04T21:22:40.914 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490885.log: gzip -5 --verbose -- 90.3% /var/log/ceph/ceph-client.admin.386484.log 2024-05-04T21:22:40.914 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.228877.log.gz 2024-05-04T21:22:40.915 INFO:teuthology.orchestra.run.smithi052.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.230501.log.gz 2024-05-04T21:22:40.915 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.490885.log.gz 2024-05-04T21:22:40.915 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.443803.log 2024-05-04T21:22:40.915 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.120561.log: /var/log/ceph/ceph-client.admin.386484.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.230097.log.gz 2024-05-04T21:22:40.915 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.232164.log 2024-05-04T21:22:40.915 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.443803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.223044.log 2024-05-04T21:22:40.915 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.231719.log.gz 2024-05-04T21:22:40.915 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.232164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.314109.log 2024-05-04T21:22:40.915 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.223044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.303969.log 2024-05-04T21:22:40.916 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.314109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.381311.log 2024-05-04T21:22:40.916 INFO:teuthology.orchestra.run.smithi052.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.281641.log.gz 2024-05-04T21:22:40.916 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.303969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79496.log 2024-05-04T21:22:40.916 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.381311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33318.log 2024-05-04T21:22:40.916 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.79496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170672.log 2024-05-04T21:22:40.916 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82585.log/var/log/ceph/ceph-client.admin.33318.log: 2024-05-04T21:22:40.916 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.170672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.292409.log 2024-05-04T21:22:40.916 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.82585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33490.log 2024-05-04T21:22:40.916 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.292409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.420615.log 2024-05-04T21:22:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.326445.log 2024-05-04T21:22:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.420615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.239700.log 2024-05-04T21:22:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.326445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186952.log 2024-05-04T21:22:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.239700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.409290.log 2024-05-04T21:22:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.186952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34846.log 2024-05-04T21:22:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.409290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.324270.log 2024-05-04T21:22:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.325558.log 2024-05-04T21:22:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.324270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.284902.log 2024-05-04T21:22:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.325558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.350301.log 2024-05-04T21:22:40.918 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.284902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96805.log 2024-05-04T21:22:40.918 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.350301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.211254.log 2024-05-04T21:22:40.918 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.96805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33994.log 2024-05-04T21:22:40.918 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2%/var/log/ceph/ceph-client.admin.211254.log: -- replaced with /var/log/ceph/ceph-client.admin.188995.log.gz 2024-05-04T21:22:40.918 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.374850.log 2024-05-04T21:22:40.918 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.374043.log 2024-05-04T21:22:40.918 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.374850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.202099.log 2024-05-04T21:22:40.918 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.374043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.462850.log 2024-05-04T21:22:40.919 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.202099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105022.log 2024-05-04T21:22:40.919 INFO:teuthology.orchestra.run.smithi052.stderr:gzip 81.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.151753.log.gz -- 2024-05-04T21:22:40.919 INFO:teuthology.orchestra.run.smithi052.stderr: /var/log/ceph/ceph-client.admin.165061.log 2024-05-04T21:22:40.919 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.462850.log: /var/log/ceph/ceph-client.admin.105022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.221391.log 2024-05-04T21:22:40.919 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.165061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.348659.log 2024-05-04T21:22:40.919 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.221391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184454.log 2024-05-04T21:22:40.919 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.348659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.316213.log 2024-05-04T21:22:40.919 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.184454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.488118.log 2024-05-04T21:22:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.316213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36481.log 2024-05-04T21:22:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.488118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.394497.log 2024-05-04T21:22:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166792.log 2024-05-04T21:22:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.394497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.419066.log 2024-05-04T21:22:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.375668.log 2024-05-04T21:22:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.419066.log: /var/log/ceph/ceph-client.admin.166792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.354727.log 2024-05-04T21:22:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.375668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491275.log 2024-05-04T21:22:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.354727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.351549.log 2024-05-04T21:22:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.491275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174723.log 2024-05-04T21:22:40.921 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.351549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35238.log 2024-05-04T21:22:40.921 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.491275.log.gz 2024-05-04T21:22:40.921 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.174723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90596.log 2024-05-04T21:22:40.921 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174326.log 2024-05-04T21:22:40.921 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.90596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37511.log 2024-05-04T21:22:40.921 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.174326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.424804.log 2024-05-04T21:22:40.921 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.0.37511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.415576.log 2024-05-04T21:22:40.921 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.424804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.300633.log 2024-05-04T21:22:40.922 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.415576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83044.log 2024-05-04T21:22:40.922 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.300633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173505.log 2024-05-04T21:22:40.922 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.83044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.466793.log 2024-05-04T21:22:40.922 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.173505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.221802.log 2024-05-04T21:22:40.922 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.466793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153476.log 2024-05-04T21:22:40.922 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.221802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35158.log 2024-05-04T21:22:40.922 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.153476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.371547.log 2024-05-04T21:22:40.922 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.463738.log 2024-05-04T21:22:40.922 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.371547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.202974.log 2024-05-04T21:22:40.923 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.463738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.307629.log 2024-05-04T21:22:40.923 INFO:teuthology.orchestra.run.smithi052.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.195924.log.gz 2024-05-04T21:22:40.923 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.202974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33502.log 2024-05-04T21:22:40.923 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.307629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.286960.log 2024-05-04T21:22:40.923 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.33502.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36507.log 2024-05-04T21:22:40.923 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.286960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107126.log 2024-05-04T21:22:40.923 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.476553.log 2024-05-04T21:22:40.923 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.394912.log 2024-05-04T21:22:40.924 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.459023.log.gz 2024-05-04T21:22:40.924 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.476553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.324736.log 2024-05-04T21:22:40.924 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.107126.log: /var/log/ceph/ceph-client.admin.394912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120154.log 2024-05-04T21:22:40.924 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.324736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.330709.log 2024-05-04T21:22:40.924 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.120154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.357812.log 2024-05-04T21:22:40.924 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.330709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.425609.log 2024-05-04T21:22:40.924 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.357812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.420209.log 2024-05-04T21:22:40.924 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.425609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.407630.log 2024-05-04T21:22:40.924 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.420209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.300227.log 2024-05-04T21:22:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.407630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.224274.log 2024-05-04T21:22:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.300227.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.350301.log.gz 2024-05-04T21:22:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.283679.log 2024-05-04T21:22:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.224274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.220993.log 2024-05-04T21:22:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.283679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.472747.log 2024-05-04T21:22:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.220993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.406340.log 2024-05-04T21:22:40.925 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.311379.log.gz 2024-05-04T21:22:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.472747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.282862.log 2024-05-04T21:22:40.926 INFO:teuthology.orchestra.run.smithi052.stderr: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.386484.log.gz 2024-05-04T21:22:40.926 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.406340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121369.log 2024-05-04T21:22:40.926 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.282862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84549.log 2024-05-04T21:22:40.926 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.121369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128545.log 2024-05-04T21:22:40.926 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.84549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.301048.log 2024-05-04T21:22:40.926 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.128545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.475694.log 2024-05-04T21:22:40.927 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.301048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.428711.log 2024-05-04T21:22:40.927 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.475694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.378548.log 2024-05-04T21:22:40.927 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.428711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171069.log 2024-05-04T21:22:40.927 INFO:teuthology.orchestra.run.smithi052.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.427332.log.gz 2024-05-04T21:22:40.927 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.378548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.419075.log 2024-05-04T21:22:40.927 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.171069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.230916.log 2024-05-04T21:22:40.927 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.419075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.437427.log 2024-05-04T21:22:40.927 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.230916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.356097.log 2024-05-04T21:22:40.927 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.437427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87058.log 2024-05-04T21:22:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.356097.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.108899.log.gz 2024-05-04T21:22:40.928 INFO:teuthology.orchestra.run.smithi052.stderr: 83.4% -- replaced with /var/log/ceph/ceph-client.admin.471462.log.gz 2024-05-04T21:22:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120145.log 2024-05-04T21:22:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.87058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172693.log 2024-05-04T21:22:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.120145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.422319.log 2024-05-04T21:22:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.172693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.298202.log 2024-05-04T21:22:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.422319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.430410.log 2024-05-04T21:22:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.298202.log: gzip -5 --verbose -- 90.4% -- replaced with /var/log/ceph/ceph-client.admin.320015.log.gz 2024-05-04T21:22:40.929 INFO:teuthology.orchestra.run.smithi052.stderr: /var/log/ceph/ceph-client.admin.150485.log 2024-05-04T21:22:40.929 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.430410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.440931.log 2024-05-04T21:22:40.929 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.150485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154384.log 2024-05-04T21:22:40.929 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.440931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.216905.log 2024-05-04T21:22:40.929 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.154384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.233418.log 2024-05-04T21:22:40.929 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.216905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178459.log 2024-05-04T21:22:40.929 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.233418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359659.log 2024-05-04T21:22:40.930 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.178459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.312693.log 2024-05-04T21:22:40.930 INFO:teuthology.orchestra.run.smithi052.stderr: 79.9% -- replaced with /var/log/ceph/ceph-client.admin.83044.log.gz 2024-05-04T21:22:40.930 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.316653.log 2024-05-04T21:22:40.930 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.359659.log: /var/log/ceph/ceph-client.admin.312693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157655.log 2024-05-04T21:22:40.930 INFO:teuthology.orchestra.run.smithi052.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.415999.log.gz 2024-05-04T21:22:40.930 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.316653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163321.log 2024-05-04T21:22:40.930 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.157655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80884.log 2024-05-04T21:22:40.930 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.163321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.218124.log 2024-05-04T21:22:40.931 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.80884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34924.log 2024-05-04T21:22:40.931 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.218124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.317478.log 2024-05-04T21:22:40.931 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.34924.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.319609.log 2024-05-04T21:22:40.931 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.317478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93081.log 2024-05-04T21:22:40.931 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.319609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.201275.log 2024-05-04T21:22:40.931 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.93081.log: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.223035.log.gz 2024-05-04T21:22:40.931 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.201275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.481699.log 90.4% -- replaced with /var/log/ceph/ceph-client.admin.223044.log.gz 2024-05-04T21:22:40.931 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-04T21:22:40.932 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.298616.log.gz 2024-05-04T21:22:40.932 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173108.log 81.8% -- replaced with /var/log/ceph/ceph-client.admin.300227.log.gz 2024-05-04T21:22:40.932 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-04T21:22:40.932 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.481699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.429992.log 2024-05-04T21:22:40.932 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.173108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.295326.log 2024-05-04T21:22:40.932 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.429992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.426529.log 2024-05-04T21:22:40.932 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.295326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.197564.log 2024-05-04T21:22:40.932 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.426529.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-04T21:22:40.933 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.197564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.402150.log 2024-05-04T21:22:40.933 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36838.log 2024-05-04T21:22:40.933 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.402150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81727.log 2024-05-04T21:22:40.933 INFO:teuthology.orchestra.run.smithi052.stderr: 79.5% -- replaced with /var/log/ceph/ceph-client.admin.405115.log.gz 2024-05-04T21:22:40.933 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.302279.log 2024-05-04T21:22:40.933 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.81727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.434968.log 2024-05-04T21:22:40.934 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.302279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36325.log 2024-05-04T21:22:40.934 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.434968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36117.log 2024-05-04T21:22:40.934 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.462850.log.gz 2024-05-04T21:22:40.934 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.445321.log 2024-05-04T21:22:40.934 INFO:teuthology.orchestra.run.smithi052.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.481261.log.gz 2024-05-04T21:22:40.934 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.483891.log.gz 2024-05-04T21:22:40.934 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.378557.log 2024-05-04T21:22:40.934 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.445321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33968.log 2024-05-04T21:22:40.934 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.378557.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.303969.log.gz 2024-05-04T21:22:40.935 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.417779.log 2024-05-04T21:22:40.935 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.198384.log 2024-05-04T21:22:40.935 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.417779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162443.log 2024-05-04T21:22:40.935 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.198384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.361646.log 2024-05-04T21:22:40.935 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.162443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35212.log 2024-05-04T21:22:40.935 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.361646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.229274.log 2024-05-04T21:22:40.935 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.356952.log 2024-05-04T21:22:40.935 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.229274.log: /var/log/ceph/ceph-client.admin.356952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.330700.log 2024-05-04T21:22:40.935 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.345476.log.gz 2024-05-04T21:22:40.936 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35903.log 2024-05-04T21:22:40.936 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.330700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.411878.log 2024-05-04T21:22:40.936 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.438461.log 2024-05-04T21:22:40.936 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.411878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.210428.log 2024-05-04T21:22:40.936 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.438461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158092.log 2024-05-04T21:22:40.936 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.210428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.479985.log 2024-05-04T21:22:40.936 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.293626.log 2024-05-04T21:22:40.937 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.158092.log: /var/log/ceph/ceph-client.admin.479985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122728.log 2024-05-04T21:22:40.937 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.307222.log 2024-05-04T21:22:40.937 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.293626.log: /var/log/ceph/ceph-client.admin.122728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.486426.log 2024-05-04T21:22:40.937 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.307222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184872.log 2024-05-04T21:22:40.937 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.486426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35054.log 2024-05-04T21:22:40.937 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.184872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129500.log 2024-05-04T21:22:40.937 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35054.log: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.175539.log.gz 2024-05-04T21:22:40.937 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.175548.log.gz 2024-05-04T21:22:40.938 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.414190.log 2024-05-04T21:22:40.938 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.129500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159014.log 2024-05-04T21:22:40.938 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.414190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.435382.log 2024-05-04T21:22:40.938 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.159014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.454546.log 2024-05-04T21:22:40.938 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.435382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33104.log 2024-05-04T21:22:40.938 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.380490.log 2024-05-04T21:22:40.938 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.454546.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.86255.log.gz 2024-05-04T21:22:40.938 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490989.log 2024-05-04T21:22:40.939 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.380490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.451291.log 2024-05-04T21:22:40.939 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109297.log 90.3% -- replaced with /var/log/ceph/ceph-client.admin.230916.log.gz 2024-05-04T21:22:40.939 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-04T21:22:40.939 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.451291.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.490989.log.gz 2024-05-04T21:22:40.939 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.306410.log 2024-05-04T21:22:40.939 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.109297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.318358.log 2024-05-04T21:22:40.939 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose 84.3% -- replaced with /var/log/ceph/ceph-client.admin.443803.log.gz 2024-05-04T21:22:40.940 INFO:teuthology.orchestra.run.smithi052.stderr: -- /var/log/ceph/ceph-client.admin.165931.log 2024-05-04T21:22:40.940 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.306410.log: /var/log/ceph/ceph-client.admin.318358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187764.log 2024-05-04T21:22:40.940 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.165931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.468925.log 2024-05-04T21:22:40.940 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.187764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.487304.log 2024-05-04T21:22:40.940 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.468925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36377.log 2024-05-04T21:22:40.940 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.487304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34228.log 2024-05-04T21:22:40.940 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.424795.log 2024-05-04T21:22:40.940 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.368670.log 2024-05-04T21:22:40.940 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.424795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.426935.log 2024-05-04T21:22:40.941 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.368670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.484336.log 2024-05-04T21:22:40.941 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.426935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.240555.log 2024-05-04T21:22:40.941 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.484336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.414748.log 2024-05-04T21:22:40.941 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.240555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167236.log 2024-05-04T21:22:40.941 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.414748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.421901.log 2024-05-04T21:22:40.941 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.167236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191153.log 2024-05-04T21:22:40.941 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.421901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.200877.log 2024-05-04T21:22:40.941 INFO:teuthology.orchestra.run.smithi052.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.302681.log.gz 2024-05-04T21:22:40.942 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.191153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.374859.log 2024-05-04T21:22:40.942 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36585.log.gz 2024-05-04T21:22:40.942 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.200877.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35641.log.gz 2024-05-04T21:22:40.942 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36455.log 2024-05-04T21:22:40.942 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.374859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.402958.log 2024-05-04T21:22:40.942 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.467217.log 2024-05-04T21:22:40.942 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.402958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127327.log 2024-05-04T21:22:40.942 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.467217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188567.log 2024-05-04T21:22:40.943 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.127327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.400026.log 2024-05-04T21:22:40.943 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.188567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359297.log 2024-05-04T21:22:40.943 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.400026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34463.log 2024-05-04T21:22:40.943 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.359297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33706.log 2024-05-04T21:22:40.943 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.0.34463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193892.log 2024-05-04T21:22:40.943 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.228471.log 2024-05-04T21:22:40.943 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.193892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.363297.log 2024-05-04T21:22:40.943 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.228471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.307638.log 2024-05-04T21:22:40.944 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.363297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.325151.log 2024-05-04T21:22:40.944 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.307638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190702.log 2024-05-04T21:22:40.944 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.325151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423539.log 2024-05-04T21:22:40.944 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.190702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.461105.log 2024-05-04T21:22:40.944 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.423539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35563.log 2024-05-04T21:22:40.944 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.461105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.220167.log 2024-05-04T21:22:40.944 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35106.log 2024-05-04T21:22:40.944 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.242765.log.gz 2024-05-04T21:22:40.944 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5/var/log/ceph/ceph-client.admin.220167.log: --verbose -- /var/log/ceph/ceph-client.admin.171890.log 2024-05-04T21:22:40.945 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.242367.log.gz 2024-05-04T21:22:40.945 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.438452.log 2024-05-04T21:22:40.945 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.171890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.299021.log 2024-05-04T21:22:40.945 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.438452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.476124.log 2024-05-04T21:22:40.945 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.299021.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.146253.log.gz 2024-05-04T21:22:40.945 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177169.log 2024-05-04T21:22:40.945 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.476124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36351.log 2024-05-04T21:22:40.945 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.177169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.486865.log 2024-05-04T21:22:40.946 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145039.log 2024-05-04T21:22:40.946 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.486865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167670.log 2024-05-04T21:22:40.946 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.145039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.373623.log 2024-05-04T21:22:40.946 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.167670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103400.log 2024-05-04T21:22:40.946 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.373623.log: 83.8% -- replaced with /var/log/ceph/ceph-client.admin.93081.log.gz 2024-05-04T21:22:40.946 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.213991.log.gz 2024-05-04T21:22:40.946 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.213150.log 2024-05-04T21:22:40.946 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.103400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.400894.log 2024-05-04T21:22:40.946 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.213150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.449810.log 2024-05-04T21:22:40.947 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.400894.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.122297.log 2024-05-04T21:22:40.947 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.449810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111354.log 2024-05-04T21:22:40.947 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.122297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78661.log 2024-05-04T21:22:40.947 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.111354.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.112216.log.gz 2024-05-04T21:22:40.947 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.78661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.369902.log 2024-05-04T21:22:40.947 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.228065.log 2024-05-04T21:22:40.947 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.369902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.342881.log 2024-05-04T21:22:40.947 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.228065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.411399.log 2024-05-04T21:22:40.947 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.342881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.226441.log 2024-05-04T21:22:40.948 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.411399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.226026.log 2024-05-04T21:22:40.948 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.226441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128536.log 2024-05-04T21:22:40.948 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.226026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87067.log 2024-05-04T21:22:40.948 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.128536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37139.log 2024-05-04T21:22:40.948 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.126383.log.gz 2024-05-04T21:22:40.948 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.87067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.209171.log 2024-05-04T21:22:40.948 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.416837.log 2024-05-04T21:22:40.948 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.37139.log: /var/log/ceph/ceph-client.admin.209171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.454952.log 2024-05-04T21:22:40.948 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.416837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.345015.log 2024-05-04T21:22:40.949 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.454952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104614.log 2024-05-04T21:22:40.949 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.345015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.344605.log 2024-05-04T21:22:40.949 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.104614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.440080.log 2024-05-04T21:22:40.949 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.344605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35428.log 2024-05-04T21:22:40.949 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.440080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36403.log 2024-05-04T21:22:40.949 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.351955.log 2024-05-04T21:22:40.949 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.469818.log 2024-05-04T21:22:40.949 INFO:teuthology.orchestra.run.smithi052.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.201275.log.gz 2024-05-04T21:22:40.950 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.351955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.477411.log 2024-05-04T21:22:40.950 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.469818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33689.log 2024-05-04T21:22:40.950 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.477411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.485574.log 2024-05-04T21:22:40.950 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.435382.log.gz 2024-05-04T21:22:40.950 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5/var/log/ceph/ceph-client.admin.33689.log: --verbose -- /var/log/ceph/ceph-client.admin.175956.log 2024-05-04T21:22:40.950 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.485574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.243993.log 2024-05-04T21:22:40.950 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.175956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109708.log 2024-05-04T21:22:40.950 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.243993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.399218.log 2024-05-04T21:22:40.950 INFO:teuthology.orchestra.run.smithi052.stderr: 89.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-04T21:22:40.951 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.109708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.367460.log 2024-05-04T21:22:40.951 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.399218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.320837.log 2024-05-04T21:22:40.951 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.367460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.471052.log 2024-05-04T21:22:40.951 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.320837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.291478.log 2024-05-04T21:22:40.951 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.471052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.418186.log 2024-05-04T21:22:40.951 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33344.log 2024-05-04T21:22:40.951 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.291478.log: /var/log/ceph/ceph-client.admin.418186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36663.log 2024-05-04T21:22:40.951 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33078.log 2024-05-04T21:22:40.952 INFO:teuthology.orchestra.run.smithi052.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.373213.log.gz 2024-05-04T21:22:40.952 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.432860.log 2024-05-04T21:22:40.952 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.379574.log 2024-05-04T21:22:40.952 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.432860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.324727.log 2024-05-04T21:22:40.952 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.379574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.399624.log 2024-05-04T21:22:40.952 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.324727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155796.log 2024-05-04T21:22:40.952 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.399624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.212540.log 2024-05-04T21:22:40.952 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.155796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.403373.log 2024-05-04T21:22:40.953 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.212540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.411408.log 2024-05-04T21:22:40.953 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.403373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.220582.log 2024-05-04T21:22:40.953 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.411408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.396957.log 2024-05-04T21:22:40.953 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.220582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.282871.log 2024-05-04T21:22:40.953 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.318358.log.gz 2024-05-04T21:22:40.953 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.396957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180177.log 2024-05-04T21:22:40.953 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.282871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194289.log 2024-05-04T21:22:40.953 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.180177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.203371.log 2024-05-04T21:22:40.954 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.194289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34950.log 2024-05-04T21:22:40.954 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.203371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.473167.log 2024-05-04T21:22:40.954 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.454147.log 2024-05-04T21:22:40.954 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.473167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173514.log 2024-05-04T21:22:40.954 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.454147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79078.log 2024-05-04T21:22:40.954 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.173514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490937.log 2024-05-04T21:22:40.955 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.79078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.294108.log 2024-05-04T21:22:40.955 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.322940.log 2024-05-04T21:22:40.955 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.490937.log.gz 2024-05-04T21:22:40.955 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.294108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.425202.log 2024-05-04T21:22:40.955 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.322940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.349055.log 2024-05-04T21:22:40.955 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.425202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.371983.log 2024-05-04T21:22:40.955 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.240555.log.gz 2024-05-04T21:22:40.955 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.209180.log 2024-05-04T21:22:40.955 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.349055.log: /var/log/ceph/ceph-client.admin.371983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.409299.log 2024-05-04T21:22:40.956 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.209180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.450657.log 2024-05-04T21:22:40.956 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.409299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126830.log 2024-05-04T21:22:40.956 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.450657.log: -- /var/log/ceph/ceph-client.admin.453335.log 2024-05-04T21:22:40.956 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.126830.log: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.307629.log.gz 2024-05-04T21:22:40.956 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34124.log 2024-05-04T21:22:40.956 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.472747.log.gz 2024-05-04T21:22:40.956 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126821.log 2024-05-04T21:22:40.957 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.453335.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.307638.log.gz 2024-05-04T21:22:40.957 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78068.log 2024-05-04T21:22:40.957 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.126821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.374034.log 2024-05-04T21:22:40.957 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.78068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.396948.log 2024-05-04T21:22:40.957 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.374034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.200460.log 2024-05-04T21:22:40.957 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.396948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.226847.log 2024-05-04T21:22:40.957 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.200460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.484327.log 2024-05-04T21:22:40.957 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.226847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152622.log 2024-05-04T21:22:40.958 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.484327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.405943.log 2024-05-04T21:22:40.958 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.152622.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32882.log.gz 2024-05-04T21:22:40.958 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32947.log.gz 2024-05-04T21:22:40.958 INFO:teuthology.orchestra.run.smithi052.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32973.log.gz 2024-05-04T21:22:40.958 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177580.log 2024-05-04T21:22:40.958 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33000.log.gz 2024-05-04T21:22:40.959 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.405943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.315807.log 2024-05-04T21:22:40.959 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.177580.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33448.log.gz 2024-05-04T21:22:40.959 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148765.log 2024-05-04T21:22:40.959 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.315807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164626.log 2024-05-04T21:22:40.959 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.148765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.474826.log 2024-05-04T21:22:40.959 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.164626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.198790.log 2024-05-04T21:22:40.959 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.474826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192936.log 2024-05-04T21:22:40.959 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.198790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.345024.log 2024-05-04T21:22:40.960 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.192936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35877.log 2024-05-04T21:22:40.960 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.345024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127318.log 91.1% 2024-05-04T21:22:40.960 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34280.log.gz 2024-05-04T21:22:40.960 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86661.log 2024-05-04T21:22:40.960 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.127318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.228056.log 2024-05-04T21:22:40.960 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.86661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.305190.log 2024-05-04T21:22:40.960 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.228056.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.440931.log.gz 2024-05-04T21:22:40.960 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.243186.log 2024-05-04T21:22:40.961 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.305190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.445312.log 2024-05-04T21:22:40.961 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.243186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.349470.log 2024-05-04T21:22:40.961 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.445312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.482996.log 2024-05-04T21:22:40.961 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.349470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36559.log 2024-05-04T21:22:40.961 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.482996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.439681.log 2024-05-04T21:22:40.961 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.469818.log.gz 2024-05-04T21:22:40.961 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.461596.log 2024-05-04T21:22:40.961 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.439681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.200469.log 2024-05-04T21:22:40.961 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.386493.log 2024-05-04T21:22:40.962 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.461596.log: /var/log/ceph/ceph-client.admin.200469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108028.log 2024-05-04T21:22:40.962 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.386493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.232604.log 2024-05-04T21:22:40.962 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.108028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.293221.log 2024-05-04T21:22:40.962 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.232604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.297806.log 2024-05-04T21:22:40.962 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.293221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186081.log 2024-05-04T21:22:40.962 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.297806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147500.log 2024-05-04T21:22:40.962 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.186081.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.475694.log.gz 2024-05-04T21:22:40.962 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.469407.log 2024-05-04T21:22:40.963 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.147500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.299831.log 2024-05-04T21:22:40.963 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.469407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.372389.log 2024-05-04T21:22:40.963 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.299831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33212.log 2024-05-04T21:22:40.963 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.372389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.367451.log 2024-05-04T21:22:40.963 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187349.log 2024-05-04T21:22:40.963 INFO:teuthology.orchestra.run.smithi052.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.402150.log.gz 2024-05-04T21:22:40.963 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.367451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.441343.log 2024-05-04T21:22:40.963 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.191153.log.gz 2024-05-04T21:22:40.964 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.187349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32781.log 2024-05-04T21:22:40.964 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.441343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.223446.log 2024-05-04T21:22:40.964 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.32781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.293212.log 2024-05-04T21:22:40.964 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.223446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.474429.log 2024-05-04T21:22:40.964 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.293212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.285320.log 2024-05-04T21:22:40.964 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.422319.log.gz 2024-05-04T21:22:40.964 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.474429.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.404177.log.gz 2024-05-04T21:22:40.964 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.383781.log 2024-05-04T21:22:40.964 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.285320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.476562.log 2024-05-04T21:22:40.965 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.383781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.384925.log 2024-05-04T21:22:40.965 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.476562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.196760.log 2024-05-04T21:22:40.965 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.177580.log.gz 2024-05-04T21:22:40.965 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.384925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.347123.log 2024-05-04T21:22:40.965 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.196760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.457805.log 2024-05-04T21:22:40.965 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.347123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.436727.log 2024-05-04T21:22:40.965 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.457805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107581.log 2024-05-04T21:22:40.965 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.436727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.223868.log 2024-05-04T21:22:40.966 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.107581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178450.log 2024-05-04T21:22:40.966 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.223868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491353.log 2024-05-04T21:22:40.966 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.178450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153037.log 2024-05-04T21:22:40.966 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.491353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.317063.log 2024-05-04T21:22:40.966 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.491353.log.gz 2024-05-04T21:22:40.966 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.153037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.479994.log 2024-05-04T21:22:40.966 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.317063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.296986.log 2024-05-04T21:22:40.967 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187358.log/var/log/ceph/ceph-client.admin.479994.log: 2024-05-04T21:22:40.967 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.296986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34335.log 2024-05-04T21:22:40.967 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.187358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.434084.log 2024-05-04T21:22:40.967 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.34335.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.180168.log 2024-05-04T21:22:40.967 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.434084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.408081.log 2024-05-04T21:22:40.967 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.180168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104216.log 2024-05-04T21:22:40.967 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.408081.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.226035.log 2024-05-04T21:22:40.967 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.104216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.456585.log 2024-05-04T21:22:40.968 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.226035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.458608.log 2024-05-04T21:22:40.968 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.456585.log: /var/log/ceph/ceph-client.admin.458608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.284099.log 2024-05-04T21:22:40.968 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.404598.log 2024-05-04T21:22:40.968 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.284099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.316204.log 2024-05-04T21:22:40.968 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36892.log 2024-05-04T21:22:40.968 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.404598.log: /var/log/ceph/ceph-client.admin.316204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172296.log 2024-05-04T21:22:40.968 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359442.log 2024-05-04T21:22:40.968 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.172296.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.168531.log 2024-05-04T21:22:40.969 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.359442.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.293212.log.gz 2024-05-04T21:22:40.969 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37076.log 2024-05-04T21:22:40.969 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.168531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.362886.log 2024-05-04T21:22:40.969 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.293221.log.gz 2024-05-04T21:22:40.969 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.37076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168968.log 2024-05-04T21:22:40.969 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.299021.log.gz 2024-05-04T21:22:40.969 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.362886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.230510.log 2024-05-04T21:22:40.969 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.168968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82134.log 2024-05-04T21:22:40.970 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.230510.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.200877.log.gz 2024-05-04T21:22:40.970 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91519.log 2024-05-04T21:22:40.970 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.82134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.470232.log 2024-05-04T21:22:40.970 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.91519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.440922.log 2024-05-04T21:22:40.970 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.470232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.316644.log 2024-05-04T21:22:40.970 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.440922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.364572.log 2024-05-04T21:22:40.970 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.316644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36786.log 2024-05-04T21:22:40.970 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.364572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98512.log 2024-05-04T21:22:40.971 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.350730.log 2024-05-04T21:22:40.971 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.98512.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-04T21:22:40.971 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.350730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161168.log 2024-05-04T21:22:40.971 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35956.log 2024-05-04T21:22:40.971 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.161168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158507.log 2024-05-04T21:22:40.971 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37327.log 2024-05-04T21:22:40.971 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.158507.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-04T21:22:40.971 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.37327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98913.log 2024-05-04T21:22:40.971 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-mds.c.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.405943.log.gz 2024-05-04T21:22:40.972 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96395.log 2024-05-04T21:22:40.972 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.98913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100148.log 2024-05-04T21:22:40.972 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.96395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.203799.log 2024-05-04T21:22:40.972 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.100148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.353634.log 2024-05-04T21:22:40.972 INFO:teuthology.orchestra.run.smithi052.stderr: 77.6% -- replaced with /var/log/ceph/ceph-client.admin.220167.log.gz 2024-05-04T21:22:40.972 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.203799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490530.log 2024-05-04T21:22:40.972 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.353634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.204219.log 2024-05-04T21:22:40.972 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490530.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.220582.log.gz 2024-05-04T21:22:40.973 INFO:teuthology.orchestra.run.smithi052.stderr: 83.5% -- replaced with /var/log/ceph/ceph-client.admin.229274.log.gz 2024-05-04T21:22:40.973 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490504.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.490530.log.gz 2024-05-04T21:22:40.973 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-04T21:22:40.973 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.204219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128852.log 2024-05-04T21:22:40.973 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.313648.log 2024-05-04T21:22:40.973 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.490504.log.gz 2024-05-04T21:22:40.974 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.128852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186943.log 2024-05-04T21:22:40.974 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.313648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161603.log 2024-05-04T21:22:40.974 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.186943.log: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.117231.log.gz 2024-05-04T21:22:40.974 INFO:teuthology.orchestra.run.smithi052.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.117638.log.gz 2024-05-04T21:22:40.974 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-04T21:22:40.974 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.161603.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.117240.log.gz 2024-05-04T21:22:40.974 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33500.log 2024-05-04T21:22:40.974 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108436.log 2024-05-04T21:22:40.975 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.233409.log 2024-05-04T21:22:40.975 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.120561.log.gz 2024-05-04T21:22:40.975 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.108436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.371134.log 2024-05-04T21:22:40.975 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.233409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.459420.log 2024-05-04T21:22:40.975 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.371134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.348244.log 2024-05-04T21:22:40.975 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.459420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.282048.log 2024-05-04T21:22:40.975 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.348244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.396121.log 2024-05-04T21:22:40.975 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.282048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359036.log 2024-05-04T21:22:40.975 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.396121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359027.log 2024-05-04T21:22:40.976 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.359036.log: gzip -5 --verbose -- 80.4% -- replaced with /var/log/ceph/ceph-client.admin.381708.log.gz 2024-05-04T21:22:40.976 INFO:teuthology.orchestra.run.smithi052.stderr: /var/log/ceph/ceph-client.admin.178894.log 2024-05-04T21:22:40.976 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.359027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101775.log 2024-05-04T21:22:40.976 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.178894.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.381311.log.gz 2024-05-04T21:22:40.976 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95169.log 2024-05-04T21:22:40.976 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.101775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34202.log 2024-05-04T21:22:40.976 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.95169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491145.log 2024-05-04T21:22:40.976 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.281231.log 2024-05-04T21:22:40.976 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.491145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.398746.log 2024-05-04T21:22:40.977 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.491145.log.gz 2024-05-04T21:22:40.977 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.281231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.410581.log 2024-05-04T21:22:40.977 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.398746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.320006.log 2024-05-04T21:22:40.977 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.410581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491379.log 2024-05-04T21:22:40.977 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.320006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.286549.log 2024-05-04T21:22:40.977 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.491379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103807.log 2024-05-04T21:22:40.978 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.378548.log.gz 2024-05-04T21:22:40.978 INFO:teuthology.orchestra.run.smithi052.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.491379.log.gz 2024-05-04T21:22:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.286549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.456982.log 2024-05-04T21:22:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.103807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.202539.log 2024-05-04T21:22:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.456982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.446778.log 2024-05-04T21:22:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.202539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.321648.log 2024-05-04T21:22:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.446778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.326030.log 2024-05-04T21:22:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.321648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.216896.log 2024-05-04T21:22:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.326030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.470652.log 2024-05-04T21:22:40.979 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.216896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.474835.log 2024-05-04T21:22:40.979 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.470652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.283282.log 2024-05-04T21:22:40.979 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.474835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.465035.log 2024-05-04T21:22:40.979 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.283282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33890.log 2024-05-04T21:22:40.979 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.465035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.206353.log 2024-05-04T21:22:40.979 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.287366.log 2024-05-04T21:22:40.979 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.206353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.372796.log 2024-05-04T21:22:40.979 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.287366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359336.log 2024-05-04T21:22:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.372796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.282057.log 2024-05-04T21:22:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.359336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80435.log 2024-05-04T21:22:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.282057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.405528.log 2024-05-04T21:22:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.80435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.241509.log 2024-05-04T21:22:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.405528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.433263.log 2024-05-04T21:22:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.241509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.304783.log 2024-05-04T21:22:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.433263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.332140.log 2024-05-04T21:22:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.304783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.411000.log 2024-05-04T21:22:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.332140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.204684.log 2024-05-04T21:22:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.411000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109306.log 2024-05-04T21:22:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.204684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.422725.log 2024-05-04T21:22:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.109306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156428.log 2024-05-04T21:22:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.422725.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34335.log.gz 2024-05-04T21:22:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188576.log 2024-05-04T21:22:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.156428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.432035.log 2024-05-04T21:22:40.982 INFO:teuthology.orchestra.run.smithi052.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.34463.log.gz 2024-05-04T21:22:40.982 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.188576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.361235.log 2024-05-04T21:22:40.982 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.432035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.418666.log 2024-05-04T21:22:40.982 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.361235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.312232.log 2024-05-04T21:22:40.982 INFO:teuthology.orchestra.run.smithi052.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36507.log.gz 2024-05-04T21:22:40.982 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.418666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111363.log 2024-05-04T21:22:40.982 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.312232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.198781.log 2024-05-04T21:22:40.982 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.111363.log: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.466793.log.gz 2024-05-04T21:22:40.982 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.317921.log 2024-05-04T21:22:40.983 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.198781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.478277.log 2024-05-04T21:22:40.983 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.317921.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33344.log.gz 2024-05-04T21:22:40.983 INFO:teuthology.orchestra.run.smithi052.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33890.log.gz 2024-05-04T21:22:40.983 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.215201.log 2024-05-04T21:22:40.983 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.478277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.485583.log 2024-05-04T21:22:40.983 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.215201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.371556.log 2024-05-04T21:22:40.984 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.485583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44704.log 2024-05-04T21:22:40.984 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164191.log 2024-05-04T21:22:40.984 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.371556.log: /var/log/ceph/ceph-client.admin.44704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.322931.log 2024-05-04T21:22:40.984 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.164191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.317072.log 2024-05-04T21:22:40.984 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.322931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.364983.log 2024-05-04T21:22:40.984 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.317072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.366229.log 2024-05-04T21:22:40.984 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.364983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.196751.log 2024-05-04T21:22:40.984 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.366229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.378137.log 2024-05-04T21:22:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.196751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.214397.log 2024-05-04T21:22:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.378137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191162.log 2024-05-04T21:22:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.214397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.313163.log 2024-05-04T21:22:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.191162.log: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.111354.log.gz 2024-05-04T21:22:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.317469.log 2024-05-04T21:22:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.313163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.310982.log 2024-05-04T21:22:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.317469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.314985.log 2024-05-04T21:22:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.310982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191599.log 2024-05-04T21:22:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.314985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.479136.log 2024-05-04T21:22:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.191599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.385266.log 2024-05-04T21:22:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.479136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184466.log 2024-05-04T21:22:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.385266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.280825.log 2024-05-04T21:22:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.184466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33499.log 2024-05-04T21:22:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.280825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.349899.log 2024-05-04T21:22:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.456991.log 2024-05-04T21:22:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.349899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.343762.log 2024-05-04T21:22:40.987 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.456991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.382974.log 2024-05-04T21:22:40.987 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36892.log.gz 2024-05-04T21:22:40.987 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.343762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161612.log 2024-05-04T21:22:40.987 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.382974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153908.log 2024-05-04T21:22:40.987 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.161612.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.100148.log.gz 2024-05-04T21:22:40.987 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.206787.log 2024-05-04T21:22:40.987 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.153908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.240564.log 2024-05-04T21:22:40.987 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.206787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.320431.log 2024-05-04T21:22:40.987 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.240564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.243592.log 2024-05-04T21:22:40.988 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.320431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.406800.log 2024-05-04T21:22:40.988 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.243592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.221811.log 2024-05-04T21:22:40.988 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.406800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172702.log 2024-05-04T21:22:40.988 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.221811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.465026.log 2024-05-04T21:22:40.988 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.172702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154805.log 2024-05-04T21:22:40.988 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.465026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35480.log 2024-05-04T21:22:40.988 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.154805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.364563.log 2024-05-04T21:22:40.988 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.214803.log 2024-05-04T21:22:40.989 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36637.log 2024-05-04T21:22:40.989 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.364563.log: /var/log/ceph/ceph-client.admin.214803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.205505.log 2024-05-04T21:22:40.989 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.373632.log 2024-05-04T21:22:40.989 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.205505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.198375.log 2024-05-04T21:22:40.989 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.373632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.471877.log 2024-05-04T21:22:40.989 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.198375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.319186.log 2024-05-04T21:22:40.989 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.471877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36169.log 2024-05-04T21:22:40.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.319186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.229283.log 2024-05-04T21:22:40.990 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.297806.log.gz 2024-05-04T21:22:40.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.459429.log 2024-05-04T21:22:40.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.229283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.430816.log 2024-05-04T21:22:40.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.459429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.464144.log 2024-05-04T21:22:40.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.430816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36008.log 2024-05-04T21:22:40.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.464144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.354736.log 2024-05-04T21:22:40.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.323396.log 2024-05-04T21:22:40.991 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.354736.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.233825.log 2024-05-04T21:22:40.991 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.323396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193445.log 2024-05-04T21:22:40.991 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.233825.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.405537.log 2024-05-04T21:22:40.991 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.118467.log.gz 2024-05-04T21:22:40.991 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.193445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98503.log 2024-05-04T21:22:40.991 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.405537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.437170.log 2024-05-04T21:22:40.991 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.98503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.234699.log 2024-05-04T21:22:40.991 INFO:teuthology.orchestra.run.smithi052.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.367451.log.gz 2024-05-04T21:22:40.992 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.437170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.483453.log 2024-05-04T21:22:40.992 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.234699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.286540.log 2024-05-04T21:22:40.992 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.483453.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.453335.log.gz 2024-05-04T21:22:40.992 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.396527.log 2024-05-04T21:22:40.992 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.286540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.315391.log 2024-05-04T21:22:40.992 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.396527.log: gzip 81.7% -- replaced with /var/log/ceph/ceph-client.admin.395309.log.gz 2024-05-04T21:22:40.992 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.489030.log 2024-05-04T21:22:40.992 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.362886.log.gz 2024-05-04T21:22:40.992 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.315391.log: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.461596.log.gz 2024-05-04T21:22:40.993 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.325567.log 2024-05-04T21:22:40.993 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.445312.log.gz 2024-05-04T21:22:40.993 INFO:teuthology.orchestra.run.smithi052.stderr: 77.2% -- replaced with /var/log/ceph/ceph-client.admin.448318.log.gz 2024-05-04T21:22:40.993 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.489030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.295335.log 2024-05-04T21:22:40.993 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.325567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.440494.log 2024-05-04T21:22:40.993 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.295335.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.421438.log 2024-05-04T21:22:40.993 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.440494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.482118.log 2024-05-04T21:22:40.994 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.421438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.314118.log 2024-05-04T21:22:40.994 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.482118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33493.log 2024-05-04T21:22:40.994 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.314118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.409696.log 2024-05-04T21:22:40.994 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.356088.log 2024-05-04T21:22:40.994 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.409696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182635.log 2024-05-04T21:22:40.994 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.356088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.233011.log 2024-05-04T21:22:40.994 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.182635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.296172.log 2024-05-04T21:22:40.994 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.456585.log.gz 2024-05-04T21:22:40.994 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.456991.log.gz 2024-05-04T21:22:40.995 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.366238.log 2024-05-04T21:22:40.995 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.233011.log: /var/log/ceph/ceph-client.admin.296172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490809.log 2024-05-04T21:22:40.995 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.366238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34898.log 2024-05-04T21:22:40.995 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.324279.log 2024-05-04T21:22:40.995 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.490809.log.gz 2024-05-04T21:22:40.995 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.362468.log 2024-05-04T21:22:40.995 INFO:teuthology.orchestra.run.smithi052.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.396527.log.gz 2024-05-04T21:22:40.995 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.324279.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.356097.log.gz 2024-05-04T21:22:40.996 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165487.log 2024-05-04T21:22:40.996 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.362468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34742.log 2024-05-04T21:22:40.996 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.165487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.385046.log 2024-05-04T21:22:40.996 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.308458.log 2024-05-04T21:22:40.996 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.371992.log 2024-05-04T21:22:40.996 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.385046.log: /var/log/ceph/ceph-client.admin.308458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190244.log 2024-05-04T21:22:40.996 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.371992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121824.log 2024-05-04T21:22:40.996 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.190244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33942.log 2024-05-04T21:22:40.996 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.121824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.384186.log 2024-05-04T21:22:40.997 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.385668.log 2024-05-04T21:22:40.997 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.223868.log.gz 2024-05-04T21:22:40.997 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.384186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35693.log 2024-05-04T21:22:40.997 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.385668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170262.log 2024-05-04T21:22:40.997 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158498.log 2024-05-04T21:22:40.997 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.170262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491197.log 2024-05-04T21:22:40.997 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.158498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.376083.log 2024-05-04T21:22:40.997 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.491197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.289411.log 2024-05-04T21:22:40.997 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4%/var/log/ceph/ceph-client.admin.376083.log: -- replaced with /var/log/ceph/ceph-client.admin.491197.log.gz 2024-05-04T21:22:40.998 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.363708.log 2024-05-04T21:22:40.998 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.289411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186546.log 2024-05-04T21:22:40.998 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.363708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.364149.log 2024-05-04T21:22:40.998 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.186546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.480832.log 2024-05-04T21:22:40.998 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.364149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.216494.log 2024-05-04T21:22:40.998 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.480832.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.422725.log.gz 2024-05-04T21:22:40.998 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99332.log 2024-05-04T21:22:40.998 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.216494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99323.log 2024-05-04T21:22:40.998 INFO:teuthology.orchestra.run.smithi052.stderr: 77.3% -- replaced with /var/log/ceph/ceph-client.admin.86246.log.gz 2024-05-04T21:22:40.999 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.99332.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.410991.log 2024-05-04T21:22:40.999 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.99323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.326866.log 2024-05-04T21:22:40.999 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.410991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.434977.log 2024-05-04T21:22:40.999 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.326866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.427837.log 2024-05-04T21:22:40.999 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.434977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.280834.log 2024-05-04T21:22:40.999 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.427837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148330.log 2024-05-04T21:22:40.999 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.280834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.295741.log 2024-05-04T21:22:40.999 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.148330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91510.log 2024-05-04T21:22:40.999 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.295741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103798.log 2024-05-04T21:22:41.000 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.91510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.320421.log 2024-05-04T21:22:41.000 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.103798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171078.log 2024-05-04T21:22:41.000 INFO:teuthology.orchestra.run.smithi052.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.185675.log.gz 2024-05-04T21:22:41.000 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.320421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.459836.log 2024-05-04T21:22:41.000 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.171078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.437418.log 2024-05-04T21:22:41.000 INFO:teuthology.orchestra.run.smithi052.stderr: 79.7% -- replaced with /var/log/ceph/ceph-client.admin.186943.log.gz 2024-05-04T21:22:41.000 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.459836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35929.log 2024-05-04T21:22:41.000 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.437418.log: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.98503.log.gz 2024-05-04T21:22:41.001 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170253.log 2024-05-04T21:22:41.001 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91057.log 2024-05-04T21:22:41.001 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.170253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.227253.log 2024-05-04T21:22:41.001 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.91057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.205090.log 2024-05-04T21:22:41.001 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.227253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490608.log 2024-05-04T21:22:41.001 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.205090.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.361235.log.gz 2024-05-04T21:22:41.001 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.198375.log.gz 2024-05-04T21:22:41.001 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.198781.log.gz 2024-05-04T21:22:41.001 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.299822.log 2024-05-04T21:22:41.002 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.168531.log.gz 2024-05-04T21:22:41.002 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.197166.log.gz 2024-05-04T21:22:41.002 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.447587.log 2024-05-04T21:22:41.002 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.168968.log.gz 2024-05-04T21:22:41.002 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.490608.log.gz 2024-05-04T21:22:41.002 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.299822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.473991.log 2024-05-04T21:22:41.002 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.447587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189807.log 2024-05-04T21:22:41.002 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.473991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.462859.log 2024-05-04T21:22:41.002 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.189807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490072.log 2024-05-04T21:22:41.003 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.462859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.210437.log 2024-05-04T21:22:41.003 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.171078.log.gz 2024-05-04T21:22:41.003 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490072.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.490072.log.gz 2024-05-04T21:22:41.003 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157249.log 2024-05-04T21:22:41.003 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.210437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110547.log 2024-05-04T21:22:41.003 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.157249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.437635.log 2024-05-04T21:22:41.003 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.110547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.363717.log 2024-05-04T21:22:41.003 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.437635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.383385.log 2024-05-04T21:22:41.004 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.363717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155805.log 2024-05-04T21:22:41.004 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.383385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.289807.log 2024-05-04T21:22:41.004 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.155805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.239709.log 2024-05-04T21:22:41.004 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.289807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.484743.log 2024-05-04T21:22:41.004 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.227253.log.gz 2024-05-04T21:22:41.004 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.239709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.353228.log 2024-05-04T21:22:41.004 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.484743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108890.log 2024-05-04T21:22:41.004 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.353228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.305597.log 2024-05-04T21:22:41.004 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.108890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116768.log 2024-05-04T21:22:41.005 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.305597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.212111.log 2024-05-04T21:22:41.005 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.116768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.349064.log 2024-05-04T21:22:41.005 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.212111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.306825.log 2024-05-04T21:22:41.005 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.349064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.372805.log 2024-05-04T21:22:41.005 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.306825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129944.log 2024-05-04T21:22:41.005 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.372805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.468451.log 2024-05-04T21:22:41.005 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.129944.log: gzip 81.3% -- replaced with /var/log/ceph/ceph-client.admin.78068.log.gz 2024-05-04T21:22:41.005 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.103391.log 2024-05-04T21:22:41.005 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.78661.log.gz 2024-05-04T21:22:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.468451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.318772.log 2024-05-04T21:22:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.103391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.466344.log 2024-05-04T21:22:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.318772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.321233.log 2024-05-04T21:22:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.466344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80893.log 2024-05-04T21:22:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.321233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.468028.log 2024-05-04T21:22:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.80893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36866.log 2024-05-04T21:22:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.468028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113054.log 2024-05-04T21:22:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33710.log 2024-05-04T21:22:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.113054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33705.log 2024-05-04T21:22:41.007 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36299.log 2024-05-04T21:22:41.007 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.299426.log 2024-05-04T21:22:41.007 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.202090.log 2024-05-04T21:22:41.007 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.299426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.463332.log 2024-05-04T21:22:41.007 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.202090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.288997.log 2024-05-04T21:22:41.007 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.463332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.380897.log 2024-05-04T21:22:41.007 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.288997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164182.log 2024-05-04T21:22:41.007 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.380897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.437644.log 2024-05-04T21:22:41.007 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.164182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175947.log 2024-05-04T21:22:41.008 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.437644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189798.log 2024-05-04T21:22:41.008 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.175947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160307.log 2024-05-04T21:22:41.008 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.189798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118059.log 2024-05-04T21:22:41.008 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.160307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.231322.log 2024-05-04T21:22:41.008 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.118059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97243.log 2024-05-04T21:22:41.008 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.231322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94356.log 2024-05-04T21:22:41.008 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.97243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192927.log 2024-05-04T21:22:41.008 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.94356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.481270.log 2024-05-04T21:22:41.009 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.192927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101369.log 2024-05-04T21:22:41.009 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.376489.log 2024-05-04T21:22:41.009 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34046.log.gz 2024-05-04T21:22:41.009 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33968.log.gz 2024-05-04T21:22:41.009 INFO:teuthology.orchestra.run.smithi052.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33994.log.gz 89.1%/var/log/ceph/ceph-client.admin.101369.log: 2024-05-04T21:22:41.009 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34020.log.gz 2024-05-04T21:22:41.009 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.481270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.294117.log 2024-05-04T21:22:41.009 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.376489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.362048.log 2024-05-04T21:22:41.009 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.294117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.209619.log 2024-05-04T21:22:41.010 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.362048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.241951.log 2024-05-04T21:22:41.010 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.209619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171475.log 2024-05-04T21:22:41.010 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.241951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186090.log 2024-05-04T21:22:41.010 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.171475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.396130.log 2024-05-04T21:22:41.010 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.186090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.413784.log 2024-05-04T21:22:41.010 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.396130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86652.log 2024-05-04T21:22:41.010 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.413784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.408478.log 2024-05-04T21:22:41.010 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.86652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.201284.log 2024-05-04T21:22:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.408478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.432026.log 2024-05-04T21:22:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.201284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99742.log 2024-05-04T21:22:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.432026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.342872.log 2024-05-04T21:22:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.99742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.429530.log 2024-05-04T21:22:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.342872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102172.log 2024-05-04T21:22:41.011 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.172296.log.gz 2024-05-04T21:22:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.429530.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.173099.log.gz 2024-05-04T21:22:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91048.log 2024-05-04T21:22:41.011 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.172702.log.gz 2024-05-04T21:22:41.012 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.102172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.344596.log 2024-05-04T21:22:41.012 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.91048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.408884.log 2024-05-04T21:22:41.012 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.344596.log: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.372389.log.gz 2024-05-04T21:22:41.012 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.422716.log 2024-05-04T21:22:41.012 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.408884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.449038.log 2024-05-04T21:22:41.012 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.422716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36273.log 2024-05-04T21:22:41.012 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.449038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.436193.log 2024-05-04T21:22:41.012 INFO:teuthology.orchestra.run.smithi052.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.152622.log.gz 2024-05-04T21:22:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33370.log 2024-05-04T21:22:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.436193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.429108.log 2024-05-04T21:22:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33052.log 2024-05-04T21:22:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.429108.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.154393.log.gz 2024-05-04T21:22:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.224710.log 2024-05-04T21:22:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.469398.log 2024-05-04T21:22:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.224710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.287357.log 2024-05-04T21:22:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.469398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.202965.log 2024-05-04T21:22:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.287357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.398265.log 2024-05-04T21:22:41.014 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.202965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.453326.log 2024-05-04T21:22:41.014 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.398265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.416406.log 2024-05-04T21:22:41.014 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.453326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.429117.log 2024-05-04T21:22:41.014 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.416406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.406791.log 2024-05-04T21:22:41.014 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.429117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.486417.log 2024-05-04T21:22:41.014 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.406791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491093.log 2024-05-04T21:22:41.014 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.486417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.218942.log 2024-05-04T21:22:41.014 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.491093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.431628.log 2024-05-04T21:22:41.015 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.491093.log.gz 2024-05-04T21:22:41.015 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.218942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.332131.log 2024-05-04T21:22:41.015 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.431628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127724.log 2024-05-04T21:22:41.015 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.332131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.217718.log 2024-05-04T21:22:41.015 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.127724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.295732.log 2024-05-04T21:22:41.015 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.217718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.454961.log 2024-05-04T21:22:41.015 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.295732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164617.log 2024-05-04T21:22:41.015 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.454961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37165.log 2024-05-04T21:22:41.015 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.164617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.428702.log 2024-05-04T21:22:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.37165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.304774.log 2024-05-04T21:22:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.428702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95981.log 2024-05-04T21:22:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.304774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.304376.log 2024-05-04T21:22:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.95981.log: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.107117.log.gz 2024-05-04T21:22:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.362057.log 2024-05-04T21:22:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.304376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.429521.log 2024-05-04T21:22:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.362057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.408893.log 2024-05-04T21:22:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.429521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90587.log 2024-05-04T21:22:41.017 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.408893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.353219.log 2024-05-04T21:22:41.017 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.90587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79980.log 2024-05-04T21:22:41.017 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.353219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.299012.log 2024-05-04T21:22:41.017 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.79980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.435373.log 2024-05-04T21:22:41.017 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.299012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.317912.log 2024-05-04T21:22:41.017 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.435373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359287.log 2024-05-04T21:22:41.017 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.317912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.284505.log 2024-05-04T21:22:41.017 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.127318.log.gz 2024-05-04T21:22:41.017 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.359287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108445.log 2024-05-04T21:22:41.018 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.284505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.371143.log 2024-05-04T21:22:41.018 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.108445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.381302.log 2024-05-04T21:22:41.018 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.371143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491406.log 2024-05-04T21:22:41.018 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.381302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.346289.log 2024-05-04T21:22:41.018 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.491406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102985.log 2024-05-04T21:22:41.018 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.491406.log.gz 2024-05-04T21:22:41.018 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.346289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81302.log 2024-05-04T21:22:41.018 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.102985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.401730.log 2024-05-04T21:22:41.018 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.81302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359433.log 2024-05-04T21:22:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.401730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.327345.log 2024-05-04T21:22:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.359433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.205099.log 2024-05-04T21:22:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.327345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.213982.log 2024-05-04T21:22:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.205099.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34658.log.gz 2024-05-04T21:22:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.414181.log 2024-05-04T21:22:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.213982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105031.log 2024-05-04T21:22:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.414181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.402552.log 2024-05-04T21:22:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.105031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173920.log 2024-05-04T21:22:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.402552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.427827.log 2024-05-04T21:22:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.173920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.222222.log 2024-05-04T21:22:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.427827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.399615.log 2024-05-04T21:22:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.222222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.296581.log 2024-05-04T21:22:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.399615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.367857.log 2024-05-04T21:22:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.296581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.482127.log 2024-05-04T21:22:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.367857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.323866.log 2024-05-04T21:22:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.482127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102578.log 2024-05-04T21:22:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.323866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423133.log 2024-05-04T21:22:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.102578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78077.log 2024-05-04T21:22:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.423133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.426926.log 2024-05-04T21:22:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.78077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100139.log 2024-05-04T21:22:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.426926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491301.log 2024-05-04T21:22:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.100139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.204693.log 2024-05-04T21:22:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.491301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.482547.log 2024-05-04T21:22:41.021 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.491301.log.gz 2024-05-04T21:22:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.204693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144633.log 2024-05-04T21:22:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.482547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156834.log 2024-05-04T21:22:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.144633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157240.log 2024-05-04T21:22:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.156834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33422.log 2024-05-04T21:22:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.157240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.215210.log 2024-05-04T21:22:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37301.log 2024-05-04T21:22:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.215210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105431.log 2024-05-04T21:22:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.37301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129509.log 2024-05-04T21:22:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.105431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.226838.log 2024-05-04T21:22:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.129509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.370312.log 2024-05-04T21:22:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.226838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.322461.log 2024-05-04T21:22:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.370312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.220573.log 2024-05-04T21:22:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.322461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.345467.log 2024-05-04T21:22:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.220573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95575.log 2024-05-04T21:22:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.345467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.356507.log 2024-05-04T21:22:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.95575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.312702.log 2024-05-04T21:22:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.356507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120967.log 2024-05-04T21:22:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.312702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33240.log 2024-05-04T21:22:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.120967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192464.log 2024-05-04T21:22:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.454555.log 2024-05-04T21:22:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.192464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.457399.log 2024-05-04T21:22:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.454555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100961.log 2024-05-04T21:22:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.457399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423936.log 2024-05-04T21:22:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.100961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.473158.log 2024-05-04T21:22:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.423936.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34794.log.gz 2024-05-04T21:22:41.024 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.99323.log.gz 2024-05-04T21:22:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.471868.log 2024-05-04T21:22:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.473158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37104.log 2024-05-04T21:22:41.025 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.99332.log.gz 2024-05-04T21:22:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.471868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.326436.log 2024-05-04T21:22:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.37104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.311794.log 2024-05-04T21:22:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.326436.log: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.100139.log.gz 2024-05-04T21:22:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.205496.log 2024-05-04T21:22:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.311794.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.99742.log.gz 2024-05-04T21:22:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.483444.log 2024-05-04T21:22:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.205496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.462452.log 2024-05-04T21:22:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.483444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94771.log 2024-05-04T21:22:41.026 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.101369.log.gz 2024-05-04T21:22:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.462452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.468916.log 2024-05-04T21:22:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.94771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.297797.log 2024-05-04T21:22:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.468916.log: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.95981.log.gz 2024-05-04T21:22:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35375.log 2024-05-04T21:22:41.026 INFO:teuthology.orchestra.run.smithi052.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.121824.log.gz 2024-05-04T21:22:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.297797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.448328.log 2024-05-04T21:22:41.027 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35375.log: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.317912.log.gz 2024-05-04T21:22:41.027 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173911.log 2024-05-04T21:22:41.027 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.448328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147084.log 2024-05-04T21:22:41.027 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.173911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.379583.log 2024-05-04T21:22:41.027 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.147084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.463729.log 2024-05-04T21:22:41.027 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.379583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125885.log 2024-05-04T21:22:41.027 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.463729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.308467.log 2024-05-04T21:22:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.125885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33266.log 2024-05-04T21:22:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.308467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.203790.log 2024-05-04T21:22:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.33266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.410572.log 2024-05-04T21:22:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.203790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121378.log 2024-05-04T21:22:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.410572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.284496.log 2024-05-04T21:22:41.028 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.148330.log.gz 2024-05-04T21:22:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.121378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.479127.log 2024-05-04T21:22:41.028 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.429521.log.gz 2024-05-04T21:22:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.284496.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.429530.log.gz 2024-05-04T21:22:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.287762.log 2024-05-04T21:22:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.479127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.399209.log 2024-05-04T21:22:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.287762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.471061.log 2024-05-04T21:22:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.399209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.233002.log 2024-05-04T21:22:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:gzip/var/log/ceph/ceph-client.admin.471061.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.94365.log 2024-05-04T21:22:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.233002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.441780.log 2024-05-04T21:22:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.94365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168540.log 2024-05-04T21:22:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.441780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.229681.log 2024-05-04T21:22:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.168540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.465907.log 2024-05-04T21:22:41.030 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.229681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36611.log 2024-05-04T21:22:41.030 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.465907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.328331.log 2024-05-04T21:22:41.030 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.232595.log 2024-05-04T21:22:41.030 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.328331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.395724.log 2024-05-04T21:22:41.030 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.232595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160298.log 2024-05-04T21:22:41.030 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.395724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93525.log 2024-05-04T21:22:41.030 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.160298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.476133.log 2024-05-04T21:22:41.030 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.93525.log: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.483444.log.gz 2024-05-04T21:22:41.030 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.489021.log 2024-05-04T21:22:41.031 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.476133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.301454.log 2024-05-04T21:22:41.031 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.489021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491067.log 2024-05-04T21:22:41.031 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.301454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.34608.log 2024-05-04T21:22:41.031 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.491067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.224265.log 2024-05-04T21:22:41.031 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.2.34608.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.491067.log.gz 2024-05-04T21:22:41.031 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.403780.log 2024-05-04T21:22:41.031 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.224265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.424342.log 2024-05-04T21:22:41.031 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.403780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145847.log 2024-05-04T21:22:41.031 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36429.log.gz 2024-05-04T21:22:41.032 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.424342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37353.log 2024-05-04T21:22:41.032 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.145847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.420624.log 2024-05-04T21:22:41.032 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.37353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85839.log 2024-05-04T21:22:41.032 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.420624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.375677.log 2024-05-04T21:22:41.032 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.85839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.377325.log 2024-05-04T21:22:41.032 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.375677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.468037.log 2024-05-04T21:22:41.032 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.377325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.361226.log 2024-05-04T21:22:41.032 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.468037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.284090.log 2024-05-04T21:22:41.032 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.361226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35002.log 2024-05-04T21:22:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.284090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.446030.log 2024-05-04T21:22:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.458211.log 2024-05-04T21:22:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.446030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.351558.log 2024-05-04T21:22:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.458211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.294514.log 2024-05-04T21:22:41.033 INFO:teuthology.orchestra.run.smithi052.stderr: 77.7% -- replaced with /var/log/ceph/ceph-client.admin.473158.log.gz/var/log/ceph/ceph-client.admin.351558.log: 2024-05-04T21:22:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.405124.log 2024-05-04T21:22:41.033 INFO:teuthology.orchestra.run.smithi052.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.449038.log.gz 2024-05-04T21:22:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.294514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.413312.log 2024-05-04T21:22:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.405124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.239290.log 2024-05-04T21:22:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.413312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163747.log 2024-05-04T21:22:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.239290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.425618.log 2024-05-04T21:22:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.163747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.306002.log 2024-05-04T21:22:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.425618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.431619.log 2024-05-04T21:22:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.306002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.477840.log 2024-05-04T21:22:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.431619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.293617.log 2024-05-04T21:22:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.477840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.355231.log 2024-05-04T21:22:41.035 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.293617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490963.log 2024-05-04T21:22:41.035 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.355231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.487709.log 2024-05-04T21:22:41.035 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.490963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.366639.log 2024-05-04T21:22:41.035 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.490963.log.gz 2024-05-04T21:22:41.035 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.487709.log: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.423936.log.gz 2024-05-04T21:22:41.035 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36944.log 2024-05-04T21:22:41.035 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.366639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.292806.log 2024-05-04T21:22:41.035 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.36944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.440089.log 2024-05-04T21:22:41.035 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.292806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.487718.log 2024-05-04T21:22:41.036 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.440089.log: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.315391.log.gz 2024-05-04T21:22:41.036 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.462016.log 2024-05-04T21:22:41.036 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.487718.log: gzip -5 --verbose 81.1% -- -- replaced with /var/log/ceph/ceph-client.admin.437418.log.gz /var/log/ceph/ceph-client.admin.37275.log 2024-05-04T21:22:41.036 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-04T21:22:41.036 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.462016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.220984.log 2024-05-04T21:22:41.036 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.37275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34872.log 2024-05-04T21:22:41.036 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.220984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.286133.log 2024-05-04T21:22:41.036 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.468916.log.gz 2024-05-04T21:22:41.036 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.34872.log: /var/log/ceph/ceph-client.admin.286133.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.447587.log.gz 2024-05-04T21:22:41.037 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.486417.log.gz 2024-05-04T21:22:41.037 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.356088.log.gz 2024-05-04T21:22:41.037 INFO:teuthology.orchestra.run.smithi052.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.344596.log.gz 2024-05-04T21:22:41.037 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.186546.log.gz 2024-05-04T21:22:41.037 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.403780.log.gz 2024-05-04T21:22:41.037 INFO:teuthology.orchestra.run.smithi052.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.320421.log.gz 2024-05-04T21:22:41.037 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.215668.log.gz 2024-05-04T21:22:41.037 INFO:teuthology.orchestra.run.smithi052.stderr: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.220984.log.gz 2024-05-04T21:22:41.038 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.79980.log.gz 2024-05-04T21:22:41.038 INFO:teuthology.orchestra.run.smithi052.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.79487.log.gz 2024-05-04T21:22:41.038 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.294514.log.gz 2024-05-04T21:22:41.038 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.299822.log.gz 2024-05-04T21:22:41.038 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.299426.log.gz 2024-05-04T21:22:41.038 INFO:teuthology.orchestra.run.smithi052.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.326436.log.gz 2024-05-04T21:22:41.038 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.202090.log.gz 2024-05-04T21:22:41.038 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.167661.log.gz 2024-05-04T21:22:41.039 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.195518.log.gz 2024-05-04T21:22:41.039 INFO:teuthology.orchestra.run.smithi052.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.205496.log.gz 2024-05-04T21:22:41.039 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.205090.log.gz 2024-05-04T21:22:41.039 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.204693.log.gz 2024-05-04T21:22:41.039 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.205099.log.gz 2024-05-04T21:22:41.039 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33160.log.gz 2024-05-04T21:22:41.039 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35054.log.gz 2024-05-04T21:22:41.039 INFO:teuthology.orchestra.run.smithi052.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35106.log.gz 2024-05-04T21:22:41.046 INFO:teuthology.orchestra.run.smithi052.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33104.log.gz 2024-05-04T21:22:41.046 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33078.log.gz 2024-05-04T21:22:41.047 INFO:teuthology.orchestra.run.smithi052.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.182635.log.gz 2024-05-04T21:22:41.047 INFO:teuthology.orchestra.run.smithi052.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.183447.log.gz 2024-05-04T21:22:41.047 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.185278.log.gz 2024-05-04T21:22:41.047 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.110957.log.gz 2024-05-04T21:22:41.047 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.93516.log.gz 2024-05-04T21:22:41.047 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.93525.log.gz 2024-05-04T21:22:41.047 INFO:teuthology.orchestra.run.smithi052.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37275.log.gz 2024-05-04T21:22:41.047 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37301.log.gz 2024-05-04T21:22:41.047 INFO:teuthology.orchestra.run.smithi052.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.285311.log.gz 2024-05-04T21:22:41.048 INFO:teuthology.orchestra.run.smithi052.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.286133.log.gz 2024-05-04T21:22:41.048 INFO:teuthology.orchestra.run.smithi052.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.455358.log.gz 2024-05-04T21:22:41.048 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.487709.log.gz 2024-05-04T21:22:41.048 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.487718.log.gz 2024-05-04T21:22:41.048 INFO:teuthology.orchestra.run.smithi052.stderr: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.469180.log.gz 2024-05-04T21:22:41.048 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.440089.log.gz 2024-05-04T21:22:41.048 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.181831.log.gz 2024-05-04T21:22:41.048 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.404589.log.gz 2024-05-04T21:22:41.048 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.351558.log.gz 2024-05-04T21:22:41.049 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.219362.log.gz 2024-05-04T21:22:41.049 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.222213.log.gz 2024-05-04T21:22:41.049 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.439273.log.gz 2024-05-04T21:22:41.049 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.464578.log.gz 2024-05-04T21:22:41.049 INFO:teuthology.orchestra.run.smithi052.stderr: 83.5% -- replaced with /var/log/ceph/ceph-client.admin.410572.log.gz 2024-05-04T21:22:41.049 INFO:teuthology.orchestra.run.smithi052.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.118886.log.gz 2024-05-04T21:22:41.049 INFO:teuthology.orchestra.run.smithi052.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.406800.log.gz 2024-05-04T21:22:41.049 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.121378.log.gz 2024-05-04T21:22:41.049 INFO:teuthology.orchestra.run.smithi052.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.462016.log.gz 2024-05-04T21:22:41.050 INFO:teuthology.orchestra.run.smithi052.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.98096.log.gz 2024-05-04T21:22:41.050 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.197970.log.gz 2024-05-04T21:22:41.050 INFO:teuthology.orchestra.run.smithi052.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.144624.log.gz 2024-05-04T21:22:41.050 INFO:teuthology.orchestra.run.smithi052.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.165052.log.gz 2024-05-04T21:22:41.050 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.152188.log.gz 2024-05-04T21:22:41.050 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.281240.log.gz 2024-05-04T21:22:41.050 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35532.log.gz 2024-05-04T21:22:41.050 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.101360.log.gz 2024-05-04T21:22:41.050 INFO:teuthology.orchestra.run.smithi052.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.322054.log.gz 2024-05-04T21:22:41.051 INFO:teuthology.orchestra.run.smithi052.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.329778.log.gz 2024-05-04T21:22:41.051 INFO:teuthology.orchestra.run.smithi052.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.455773.log.gz 2024-05-04T21:22:41.051 INFO:teuthology.orchestra.run.smithi052.stderr: 80.0% -- replaced with /var/log/ceph/ceph-client.admin.452915.log.gz 2024-05-04T21:22:41.051 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.466802.log.gz 2024-05-04T21:22:41.051 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.346716.log.gz 2024-05-04T21:22:41.051 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.460680.log.gz 2024-05-04T21:22:41.051 INFO:teuthology.orchestra.run.smithi052.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.373222.log.gz 2024-05-04T21:22:41.051 INFO:teuthology.orchestra.run.smithi052.stderr: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.453732.log.gz 2024-05-04T21:22:41.051 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.175140.log.gz 2024-05-04T21:22:41.052 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.369893.log.gz 2024-05-04T21:22:41.052 INFO:teuthology.orchestra.run.smithi052.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.407214.log.gz 2024-05-04T21:22:41.052 INFO:teuthology.orchestra.run.smithi052.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.162452.log.gz 2024-05-04T21:22:41.052 INFO:teuthology.orchestra.run.smithi052.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.477849.log.gz 2024-05-04T21:22:41.052 INFO:teuthology.orchestra.run.smithi052.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.468442.log.gz 2024-05-04T21:22:41.052 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.201692.log.gz 2024-05-04T21:22:41.052 INFO:teuthology.orchestra.run.smithi052.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.384916.log.gz 2024-05-04T21:22:41.052 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.415990.log.gz 2024-05-04T21:22:41.052 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.477420.log.gz 2024-05-04T21:22:41.053 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.359650.log.gz 2024-05-04T21:22:41.053 INFO:teuthology.orchestra.run.smithi052.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.81718.log.gz 2024-05-04T21:22:41.053 INFO:teuthology.orchestra.run.smithi052.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.466353.log.gz 2024-05-04T21:22:41.053 INFO:teuthology.orchestra.run.smithi052.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.150911.log.gz 2024-05-04T21:22:41.053 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.151355.log.gz 2024-05-04T21:22:41.053 INFO:teuthology.orchestra.run.smithi052.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.154796.log.gz 2024-05-04T21:22:41.053 INFO:teuthology.orchestra.run.smithi052.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.129094.log.gz 2024-05-04T21:22:41.053 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.301851.log.gz 2024-05-04T21:22:41.053 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.296995.log.gz 2024-05-04T21:22:41.054 INFO:teuthology.orchestra.run.smithi052.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35375.log.gz 2024-05-04T21:22:41.054 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33026.log.gz 2024-05-04T21:22:41.054 INFO:teuthology.orchestra.run.smithi052.stderr: 78.6% -- replaced with /var/log/ceph/ceph-client.admin.110538.log.gz 2024-05-04T21:22:41.055 INFO:teuthology.orchestra.run.smithi052.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.179331.log.gz 2024-05-04T21:22:41.055 INFO:teuthology.orchestra.run.smithi052.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.211245.log.gz 2024-05-04T21:22:41.055 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.431222.log.gz 2024-05-04T21:22:41.063 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.407621.log.gz 2024-05-04T21:22:41.064 INFO:teuthology.orchestra.run.smithi052.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.461096.log.gz 2024-05-04T21:22:41.064 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.352371.log.gz 2024-05-04T21:22:41.078 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.473586.log.gz 2024-05-04T21:22:41.099 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.218133.log.gz 2024-05-04T21:22:41.099 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.218540.log.gz 2024-05-04T21:22:41.129 INFO:teuthology.orchestra.run.smithi052.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.241034.log.gz 2024-05-04T21:22:41.140 INFO:teuthology.orchestra.run.smithi111.stderr: 94.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-04T21:22:41.140 INFO:teuthology.orchestra.run.smithi052.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.155267.log.gz 2024-05-04T21:22:41.140 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.144222.log.gz 2024-05-04T21:22:41.142 INFO:teuthology.orchestra.run.smithi052.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.166357.log.gz 2024-05-04T21:22:41.143 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.162046.log.gz 2024-05-04T21:22:41.143 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.158101.log.gz 2024-05-04T21:22:41.170 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.417788.log.gz 2024-05-04T21:22:41.183 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.408072.log.gz 2024-05-04T21:22:41.194 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.85377.log.gz 2024-05-04T21:22:41.201 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36247.log.gz 2024-05-04T21:22:41.202 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36325.log.gz 2024-05-04T21:22:41.202 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36403.log.gz 2024-05-04T21:22:41.202 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37249.log.gz 2024-05-04T21:22:41.251 INFO:teuthology.orchestra.run.smithi052.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.348650.log.gz 2024-05-04T21:22:41.253 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.377740.log.gz 2024-05-04T21:22:41.253 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.414739.log.gz 2024-05-04T21:22:41.253 INFO:teuthology.orchestra.run.smithi052.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.479556.log.gz 2024-05-04T21:22:41.253 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.480414.log.gz 2024-05-04T21:22:41.254 INFO:teuthology.orchestra.run.smithi052.stderr: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.204210.log.gz 2024-05-04T21:22:41.254 INFO:teuthology.orchestra.run.smithi052.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.398737.log.gz 2024-05-04T21:22:41.254 INFO:teuthology.orchestra.run.smithi052.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.402141.log.gz 2024-05-04T21:22:41.255 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.401332.log.gz 2024-05-04T21:22:41.296 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.219769.log.gz 2024-05-04T21:22:41.299 INFO:teuthology.orchestra.run.smithi052.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.400035.log.gz 2024-05-04T21:22:41.301 INFO:teuthology.orchestra.run.smithi052.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.303097.log.gz 2024-05-04T21:22:41.303 INFO:teuthology.orchestra.run.smithi052.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.188986.log.gz 2024-05-04T21:22:41.303 INFO:teuthology.orchestra.run.smithi052.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.92673.log.gz 2024-05-04T21:22:41.304 INFO:teuthology.orchestra.run.smithi052.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.315798.log.gz 2024-05-04T21:22:41.310 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.428304.log.gz 2024-05-04T21:22:41.311 INFO:teuthology.orchestra.run.smithi052.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.178015.log.gz 2024-05-04T21:22:41.312 INFO:teuthology.orchestra.run.smithi052.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.118476.log.gz 2024-05-04T21:22:41.320 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37455.log.gz 2024-05-04T21:22:41.322 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27801.log.gz 2024-05-04T21:22:41.326 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.323857.log.gz 2024-05-04T21:22:41.327 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.322470.log.gz 2024-05-04T21:22:41.330 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.102994.log.gz 2024-05-04T21:22:41.335 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.358218.log.gz 2024-05-04T21:22:41.336 INFO:teuthology.orchestra.run.smithi052.stderr: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.413775.log.gz 2024-05-04T21:22:41.345 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.194696.log.gz 2024-05-04T21:22:41.346 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.232595.log.gz 2024-05-04T21:22:41.354 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.342459.log.gz 2024-05-04T21:22:41.354 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34072.log.gz 2024-05-04T21:22:41.355 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% 89.7% -- replaced with /var/log/ceph/ceph-client.admin.35797.log.gz -- replaced with /var/log/ceph/ceph-client.admin.36689.log.gz 2024-05-04T21:22:41.355 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-04T21:22:41.358 INFO:teuthology.orchestra.run.smithi052.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.319600.log.gz 2024-05-04T21:22:41.358 INFO:teuthology.orchestra.run.smithi052.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.321639.log.gz 2024-05-04T21:22:41.361 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.313639.log.gz 2024-05-04T21:22:41.363 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.233816.log.gz 2024-05-04T21:22:41.363 INFO:teuthology.orchestra.run.smithi052.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.244403.log.gz 2024-05-04T21:22:41.363 INFO:teuthology.orchestra.run.smithi052.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.412831.log.gz 2024-05-04T21:22:41.369 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.475264.log.gz 2024-05-04T21:22:41.370 INFO:teuthology.orchestra.run.smithi052.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.394903.log.gz 2024-05-04T21:22:41.371 INFO:teuthology.orchestra.run.smithi052.stderr: 80.1% -- replaced with /var/log/ceph/ceph-client.admin.460671.log.gz 2024-05-04T21:22:41.372 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.451300.log.gz 2024-05-04T21:22:41.372 INFO:teuthology.orchestra.run.smithi052.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.369488.log.gz 2024-05-04T21:22:41.373 INFO:teuthology.orchestra.run.smithi052.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.356498.log.gz 2024-05-04T21:22:41.373 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.346725.log.gz 2024-05-04T21:22:41.375 INFO:teuthology.orchestra.run.smithi052.stderr: 77.6% -- replaced with /var/log/ceph/ceph-client.admin.349890.log.gz 2024-05-04T21:22:41.375 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.362895.log.gz 2024-05-04T21:22:41.376 INFO:teuthology.orchestra.run.smithi052.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.385037.log.gz 2024-05-04T21:22:41.378 INFO:teuthology.orchestra.run.smithi052.stderr: 78.9% -- replaced with /var/log/ceph/ceph-client.admin.454138.log.gz 2024-05-04T21:22:41.378 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.441771.log.gz 2024-05-04T21:22:41.379 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.290221.log.gz 2024-05-04T21:22:41.379 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.415585.log.gz 2024-05-04T21:22:41.381 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.180614.log.gz 2024-05-04T21:22:41.381 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.183050.log.gz 2024-05-04T21:22:41.382 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.221400.log.gz 2024-05-04T21:22:41.391 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.329302.log.gz 2024-05-04T21:22:41.397 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.421910.log.gz 2024-05-04T21:22:41.399 INFO:teuthology.orchestra.run.smithi052.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.327345.log.gz 2024-05-04T21:22:41.408 INFO:teuthology.orchestra.run.smithi052.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.483900.log.gz 2024-05-04T21:22:41.410 INFO:teuthology.orchestra.run.smithi052.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.192473.log.gz 2024-05-04T21:22:41.412 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.206344.log.gz 2024-05-04T21:22:41.419 INFO:teuthology.orchestra.run.smithi052.stderr: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.145030.log.gz 2024-05-04T21:22:41.420 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35184.log.gz 2024-05-04T21:22:41.420 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35264.log.gz 2024-05-04T21:22:41.421 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.189401.log.gz 2024-05-04T21:22:41.435 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.150057.log.gz 2024-05-04T21:22:41.444 INFO:teuthology.orchestra.run.smithi052.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.436736.log.gz 2024-05-04T21:22:41.446 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35667.log.gz 2024-05-04T21:22:41.447 INFO:teuthology.orchestra.run.smithi052.stderr: 77.3% -- replaced with /var/log/ceph/ceph-client.admin.111810.log.gz 2024-05-04T21:22:41.447 INFO:teuthology.orchestra.run.smithi052.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35745.log.gz 2024-05-04T21:22:41.454 INFO:teuthology.orchestra.run.smithi052.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.82125.log.gz 2024-05-04T21:22:41.459 INFO:teuthology.orchestra.run.smithi052.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.243583.log.gz 2024-05-04T21:22:41.461 INFO:teuthology.orchestra.run.smithi052.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.119318.log.gz 2024-05-04T21:22:41.468 INFO:teuthology.orchestra.run.smithi052.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.291963.log.gz 2024-05-04T21:22:41.471 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.411869.log.gz 2024-05-04T21:22:41.471 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.314588.log.gz 2024-05-04T21:22:41.472 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.285731.log.gz 2024-05-04T21:22:41.474 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% 90.1% -- replaced with /var/log/ceph/ceph-client.admin.119327.log.gz 2024-05-04T21:22:41.474 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.455367.log.gz 2024-05-04T21:22:41.474 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.351964.log.gz 2024-05-04T21:22:41.475 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.354330.log.gz 2024-05-04T21:22:41.486 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.307231.log.gz 2024-05-04T21:22:41.487 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.191590.log.gz 2024-05-04T21:22:41.487 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.202530.log.gz 2024-05-04T21:22:41.488 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34176.log.gz 2024-05-04T21:22:41.488 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.218531.log.gz 2024-05-04T21:22:41.488 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.405934.log.gz 2024-05-04T21:22:41.488 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.407223.log.gz 2024-05-04T21:22:41.489 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.402561.log.gz 2024-05-04T21:22:41.493 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.356961.log.gz 2024-05-04T21:22:41.494 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.358630.log.gz 2024-05-04T21:22:41.498 INFO:teuthology.orchestra.run.smithi052.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.424351.log.gz 2024-05-04T21:22:41.503 INFO:teuthology.orchestra.run.smithi052.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.178885.log.gz 2024-05-04T21:22:41.503 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.206778.log.gz 2024-05-04T21:22:41.504 INFO:teuthology.orchestra.run.smithi052.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.199240.log.gz 2024-05-04T21:22:41.505 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.300236.log.gz 2024-05-04T21:22:41.514 INFO:teuthology.orchestra.run.smithi052.stderr: 83.7% -- replaced with /var/log/ceph/ceph-client.admin.153467.log.gz 2024-05-04T21:22:41.515 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.485991.log.gz 2024-05-04T21:22:41.536 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.159449.log.gz 2024-05-04T21:22:41.578 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36721.log.gz 2024-05-04T21:22:41.578 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.417309.log.gz 2024-05-04T21:22:41.579 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35851.log.gz 2024-05-04T21:22:41.579 INFO:teuthology.orchestra.run.smithi052.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.208750.log.gz 2024-05-04T21:22:41.579 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33501.log.gz 2024-05-04T21:22:41.580 INFO:teuthology.orchestra.run.smithi052.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.84329.log.gz 2024-05-04T21:22:41.582 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.120958.log.gz 2024-05-04T21:22:41.582 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.122288.log.gz 2024-05-04T21:22:41.583 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.310973.log.gz 2024-05-04T21:22:41.586 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.365792.log.gz 2024-05-04T21:22:41.589 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.367054.log.gz 2024-05-04T21:22:41.589 INFO:teuthology.orchestra.run.smithi052.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.438046.log.gz 2024-05-04T21:22:41.590 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.281650.log.gz 2024-05-04T21:22:41.590 INFO:teuthology.orchestra.run.smithi052.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.474420.log.gz 2024-05-04T21:22:41.591 INFO:teuthology.orchestra.run.smithi052.stderr: 79.9% -- replaced with /var/log/ceph/ceph-client.admin.330248.log.gz 2024-05-04T21:22:41.591 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.438055.log.gz 2024-05-04T21:22:41.594 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.288595.log.gz 2024-05-04T21:22:41.598 INFO:teuthology.orchestra.run.smithi052.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.377731.log.gz 2024-05-04T21:22:41.599 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.462443.log.gz 2024-05-04T21:22:41.599 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.284911.log.gz 2024-05-04T21:22:41.602 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.351145.log.gz 2024-05-04T21:22:41.602 INFO:teuthology.orchestra.run.smithi052.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.179322.log.gz 2024-05-04T21:22:41.603 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.383376.log.gz 2024-05-04T21:22:41.607 INFO:teuthology.orchestra.run.smithi052.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.470223.log.gz 2024-05-04T21:22:41.607 INFO:teuthology.orchestra.run.smithi052.stderr: 77.6% -- replaced with /var/log/ceph/ceph-client.admin.403364.log.gz 2024-05-04T21:22:41.611 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.461605.log.gz 2024-05-04T21:22:41.612 INFO:teuthology.orchestra.run.smithi052.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.374445.log.gz 2024-05-04T21:22:41.613 INFO:teuthology.orchestra.run.smithi052.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.376074.log.gz 2024-05-04T21:22:41.614 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.460262.log.gz 2024-05-04T21:22:41.614 INFO:teuthology.orchestra.run.smithi052.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.358621.log.gz 2024-05-04T21:22:41.619 INFO:teuthology.orchestra.run.smithi052.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.475273.log.gz 2024-05-04T21:22:41.620 INFO:teuthology.orchestra.run.smithi052.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.159868.log.gz 2024-05-04T21:22:41.628 INFO:teuthology.orchestra.run.smithi052.stderr: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.303960.log.gz 2024-05-04T21:22:41.631 INFO:teuthology.orchestra.run.smithi052.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.225620.log.gz 2024-05-04T21:22:41.631 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% 80.2% -- replaced with /var/log/ceph/ceph-client.admin.303559.log.gz 2024-05-04T21:22:41.631 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.168096.log.gz 2024-05-04T21:22:41.631 INFO:teuthology.orchestra.run.smithi052.stderr: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.231313.log.gz 2024-05-04T21:22:41.631 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.195111.log.gz 2024-05-04T21:22:41.634 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.146673.log.gz 2024-05-04T21:22:41.638 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.145856.log.gz 2024-05-04T21:22:41.640 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.213585.log.gz 2024-05-04T21:22:41.645 INFO:teuthology.orchestra.run.smithi052.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.217307.log.gz 2024-05-04T21:22:41.645 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.216485.log.gz 2024-05-04T21:22:41.655 INFO:teuthology.orchestra.run.smithi052.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.422310.log.gz 2024-05-04T21:22:41.655 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.421031.log.gz 2024-05-04T21:22:41.656 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.412840.log.gz 2024-05-04T21:22:41.658 INFO:teuthology.orchestra.run.smithi052.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.357359.log.gz 2024-05-04T21:22:41.659 INFO:teuthology.orchestra.run.smithi052.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.357803.log.gz 2024-05-04T21:22:41.659 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33704.log.gz 2024-05-04T21:22:41.659 INFO:teuthology.orchestra.run.smithi052.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35771.log.gz 2024-05-04T21:22:41.660 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.111819.log.gz 2024-05-04T21:22:41.660 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.109717.log.gz 2024-05-04T21:22:41.662 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34150.log.gz 2024-05-04T21:22:41.664 INFO:teuthology.orchestra.run.smithi052.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.106253.log.gz 2024-05-04T21:22:41.668 INFO:teuthology.orchestra.run.smithi052.stderr: 83.4% -- replaced with /var/log/ceph/ceph-client.admin.320828.log.gz 2024-05-04T21:22:41.669 INFO:teuthology.orchestra.run.smithi052.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.323387.log.gz 2024-05-04T21:22:41.669 INFO:teuthology.orchestra.run.smithi052.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.430807.log.gz 2024-05-04T21:22:41.673 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% 90.3% -- replaced with /var/log/ceph/ceph-client.admin.342468.log.gz 2024-05-04T21:22:41.674 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.432448.log.gz 2024-05-04T21:22:41.674 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.369086.log.gz 2024-05-04T21:22:41.675 INFO:teuthology.orchestra.run.smithi052.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.455764.log.gz 2024-05-04T21:22:41.675 INFO:teuthology.orchestra.run.smithi052.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.351136.log.gz 2024-05-04T21:22:41.677 INFO:teuthology.orchestra.run.smithi052.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.385257.log.gz 2024-05-04T21:22:41.677 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.382128.log.gz 2024-05-04T21:22:41.677 INFO:teuthology.orchestra.run.smithi052.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.415165.log.gz 2024-05-04T21:22:41.678 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.381717.log.gz 2024-05-04T21:22:41.679 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.416846.log.gz 2024-05-04T21:22:41.680 INFO:teuthology.orchestra.run.smithi052.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.444590.log.gz 2024-05-04T21:22:41.681 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.471471.log.gz 2024-05-04T21:22:41.682 INFO:teuthology.orchestra.run.smithi052.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.344173.log.gz 2024-05-04T21:22:41.683 INFO:teuthology.orchestra.run.smithi052.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.476982.log.gz 2024-05-04T21:22:41.683 INFO:teuthology.orchestra.run.smithi052.stderr: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.176355.log.gz 2024-05-04T21:22:41.684 INFO:teuthology.orchestra.run.smithi052.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.186537.log.gz 2024-05-04T21:22:41.684 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.401323.log.gz 2024-05-04T21:22:41.685 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.185684.log.gz 2024-05-04T21:22:41.687 INFO:teuthology.orchestra.run.smithi052.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.419613.log.gz 2024-05-04T21:22:41.688 INFO:teuthology.orchestra.run.smithi052.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.376480.log.gz 2024-05-04T21:22:41.690 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.223859.log.gz 2024-05-04T21:22:41.718 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.297391.log.gz 2024-05-04T21:22:41.726 INFO:teuthology.orchestra.run.smithi052.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.128130.log.gz 2024-05-04T21:22:41.726 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.128139.log.gz 2024-05-04T21:22:41.730 INFO:teuthology.orchestra.run.smithi052.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.195509.log.gz 2024-05-04T21:22:41.730 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.195915.log.gz 2024-05-04T21:22:41.730 INFO:teuthology.orchestra.run.smithi052.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.162037.log.gz 2024-05-04T21:22:41.736 INFO:teuthology.orchestra.run.smithi052.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.214388.log.gz 2024-05-04T21:22:41.736 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.152631.log.gz 2024-05-04T21:22:41.737 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.156437.log.gz 2024-05-04T21:22:41.741 INFO:teuthology.orchestra.run.smithi052.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.172287.log.gz 2024-05-04T21:22:41.745 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.326039.log.gz 2024-05-04T21:22:41.751 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36195.log.gz 2024-05-04T21:22:41.758 INFO:teuthology.orchestra.run.smithi052.stderr: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.99733.log.gz 2024-05-04T21:22:41.762 INFO:teuthology.orchestra.run.smithi052.stderr: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.106706.log.gz 2024-05-04T21:22:41.763 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.412368.log.gz 2024-05-04T21:22:41.763 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.282455.log.gz 2024-05-04T21:22:41.768 INFO:teuthology.orchestra.run.smithi052.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.306816.log.gz 2024-05-04T21:22:41.771 INFO:teuthology.orchestra.run.smithi052.stderr: 84.8% 90.2% -- replaced with /var/log/ceph/ceph-client.admin.325160.log.gz 2024-05-04T21:22:41.771 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.439672.log.gz 2024-05-04T21:22:41.772 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.452112.log.gz 2024-05-04T21:22:41.772 INFO:teuthology.orchestra.run.smithi052.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.440485.log.gz 2024-05-04T21:22:41.772 INFO:teuthology.orchestra.run.smithi052.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.467619.log.gz 2024-05-04T21:22:41.774 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.168977.log.gz 2024-05-04T21:22:41.774 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.348253.log.gz 2024-05-04T21:22:41.776 INFO:teuthology.orchestra.run.smithi052.stderr: 77.3% -- replaced with /var/log/ceph/ceph-client.admin.481690.log.gz 2024-05-04T21:22:41.776 INFO:teuthology.orchestra.run.smithi052.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.485982.log.gz 2024-05-04T21:22:41.777 INFO:teuthology.orchestra.run.smithi052.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.472756.log.gz 2024-05-04T21:22:41.777 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.400479.log.gz 2024-05-04T21:22:41.778 INFO:teuthology.orchestra.run.smithi052.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.400885.log.gz 2024-05-04T21:22:41.778 INFO:teuthology.orchestra.run.smithi052.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.219353.log.gz 2024-05-04T21:22:41.784 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.161177.log.gz 2024-05-04T21:22:41.786 INFO:teuthology.orchestra.run.smithi052.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.478706.log.gz 2024-05-04T21:22:41.789 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.479565.log.gz 2024-05-04T21:22:41.790 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.292400.log.gz 2024-05-04T21:22:41.791 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.169404.log.gz 2024-05-04T21:22:41.791 INFO:teuthology.orchestra.run.smithi052.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.222633.log.gz 2024-05-04T21:22:41.802 INFO:teuthology.orchestra.run.smithi052.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.384637.log.gz 2024-05-04T21:22:41.809 INFO:teuthology.orchestra.run.smithi052.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.147896.log.gz 2024-05-04T21:22:41.813 INFO:teuthology.orchestra.run.smithi052.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.96386.log.gz 2024-05-04T21:22:41.814 INFO:teuthology.orchestra.run.smithi052.stderr: 78.5% -- replaced with /var/log/ceph/ceph-client.admin.96796.log.gz 2024-05-04T21:22:41.814 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.314109.log.gz 2024-05-04T21:22:41.816 INFO:teuthology.orchestra.run.smithi052.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.435778.log.gz 2024-05-04T21:22:41.816 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.321242.log.gz 2024-05-04T21:22:41.820 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.289402.log.gz 2024-05-04T21:22:41.820 INFO:teuthology.orchestra.run.smithi052.stderr: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.290630.log.gz 2024-05-04T21:22:41.826 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.352362.log.gz 2024-05-04T21:22:41.827 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.382965.log.gz 2024-05-04T21:22:41.827 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.176762.log.gz 2024-05-04T21:22:41.827 INFO:teuthology.orchestra.run.smithi052.stderr: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.182228.log.gz 2024-05-04T21:22:41.828 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.182237.log.gz 2024-05-04T21:22:41.830 INFO:teuthology.orchestra.run.smithi052.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.177589.log.gz 2024-05-04T21:22:41.834 INFO:teuthology.orchestra.run.smithi052.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.301860.log.gz 2024-05-04T21:22:41.837 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.215677.log.gz 2024-05-04T21:22:41.838 INFO:teuthology.orchestra.run.smithi052.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.397793.log.gz 2024-05-04T21:22:41.838 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.394506.log.gz 2024-05-04T21:22:41.843 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.288182.log.gz 2024-05-04T21:22:41.854 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.421022.log.gz 2024-05-04T21:22:41.855 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.329311.log.gz 2024-05-04T21:22:41.860 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.174732.log.gz 2024-05-04T21:22:41.861 INFO:teuthology.orchestra.run.smithi052.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.183876.log.gz 2024-05-04T21:22:41.866 INFO:teuthology.orchestra.run.smithi052.stderr: 77.6% -- replaced with /var/log/ceph/ceph-client.admin.208741.log.gz 2024-05-04T21:22:41.872 INFO:teuthology.orchestra.run.smithi052.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.227244.log.gz 2024-05-04T21:22:41.872 INFO:teuthology.orchestra.run.smithi052.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35028.log.gz 2024-05-04T21:22:41.873 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33694.log.gz 2024-05-04T21:22:41.875 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.85848.log.gz 2024-05-04T21:22:41.882 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.225214.log.gz 2024-05-04T21:22:41.883 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.117647.log.gz 2024-05-04T21:22:41.896 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.94762.log.gz 2024-05-04T21:22:41.899 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.343753.log.gz 2024-05-04T21:22:41.899 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.318781.log.gz 2024-05-04T21:22:41.899 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.289816.log.gz 2024-05-04T21:22:41.900 INFO:teuthology.orchestra.run.smithi052.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.450894.log.gz 2024-05-04T21:22:41.901 INFO:teuthology.orchestra.run.smithi052.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.325558.log.gz 2024-05-04T21:22:41.902 INFO:teuthology.orchestra.run.smithi052.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.240131.log.gz 2024-05-04T21:22:41.905 INFO:teuthology.orchestra.run.smithi052.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.376919.log.gz 2024-05-04T21:22:41.906 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.292409.log.gz 2024-05-04T21:22:41.906 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.352785.log.gz 2024-05-04T21:22:41.909 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.165496.log.gz 2024-05-04T21:22:41.911 INFO:teuthology.orchestra.run.smithi052.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.110129.log.gz 2024-05-04T21:22:41.918 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.423124.log.gz 2024-05-04T21:22:41.929 INFO:teuthology.orchestra.run.smithi052.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.305181.log.gz 2024-05-04T21:22:41.937 INFO:teuthology.orchestra.run.smithi052.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.153028.log.gz 2024-05-04T21:22:41.977 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.27566.log.gz 2024-05-04T21:22:41.981 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35401.log.gz 2024-05-04T21:22:41.983 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33318.log.gz 2024-05-04T21:22:41.983 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33292.log.gz 2024-05-04T21:22:41.984 INFO:teuthology.orchestra.run.smithi052.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33396.log.gz 2024-05-04T21:22:41.984 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.92664.log.gz 2024-05-04T21:22:41.984 INFO:teuthology.orchestra.run.smithi052.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33490.log.gz 2024-05-04T21:22:41.990 INFO:teuthology.orchestra.run.smithi052.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.105842.log.gz 2024-05-04T21:22:41.990 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.239700.log.gz 2024-05-04T21:22:41.991 INFO:teuthology.orchestra.run.smithi052.stderr: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.354727.log.gz 2024-05-04T21:22:41.992 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.348659.log.gz 2024-05-04T21:22:41.996 INFO:teuthology.orchestra.run.smithi052.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.374850.log.gz 2024-05-04T21:22:41.997 INFO:teuthology.orchestra.run.smithi052.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.375668.log.gz 2024-05-04T21:22:41.999 INFO:teuthology.orchestra.run.smithi052.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.184454.log.gz 2024-05-04T21:22:42.001 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.186952.log.gz 2024-05-04T21:22:42.012 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.92257.log.gz 2024-05-04T21:22:42.041 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.173505.log.gz 2024-05-04T21:22:42.041 INFO:teuthology.orchestra.run.smithi052.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.174723.log.gz 2024-05-04T21:22:42.042 INFO:teuthology.orchestra.run.smithi052.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35080.log.gz 2024-05-04T21:22:42.042 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35238.log.gz 2024-05-04T21:22:42.042 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.174326.log.gz 2024-05-04T21:22:42.049 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.84558.log.gz 2024-05-04T21:22:42.050 INFO:teuthology.orchestra.run.smithi052.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34513.log.gz 2024-05-04T21:22:42.054 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.424804.log.gz 2024-05-04T21:22:42.054 INFO:teuthology.orchestra.run.smithi052.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27683.log.gz 2024-05-04T21:22:42.059 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.211254.log.gz 2024-05-04T21:22:42.063 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.374043.log.gz 2024-05-04T21:22:42.064 INFO:teuthology.orchestra.run.smithi052.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.368263.log.gz 2024-05-04T21:22:42.067 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34716.log.gz 2024-05-04T21:22:42.068 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34820.log.gz 2024-05-04T21:22:42.070 INFO:teuthology.orchestra.run.smithi052.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.82585.log.gz 2024-05-04T21:22:42.072 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.97697.log.gz 2024-05-04T21:22:42.102 INFO:teuthology.orchestra.run.smithi052.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36481.log.gz 2024-05-04T21:22:42.107 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.84549.log.gz 2024-05-04T21:22:42.107 INFO:teuthology.orchestra.run.smithi052.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.87058.log.gz 2024-05-04T21:22:42.110 INFO:teuthology.orchestra.run.smithi052.stderr: 79.7% -- replaced with /var/log/ceph/ceph-client.admin.243177.log.gz 2024-05-04T21:22:42.130 INFO:teuthology.orchestra.run.smithi052.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.284902.log.gz 2024-05-04T21:22:42.135 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.463738.log.gz 2024-05-04T21:22:42.139 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.178024.log.gz 2024-05-04T21:22:42.140 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.318367.log.gz 2024-05-04T21:22:42.142 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.376928.log.gz 2024-05-04T21:22:42.142 INFO:teuthology.orchestra.run.smithi052.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.415576.log.gz 2024-05-04T21:22:42.145 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.216083.log.gz 2024-05-04T21:22:42.145 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.218124.log.gz 2024-05-04T21:22:42.145 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.192036.log.gz 2024-05-04T21:22:42.146 INFO:teuthology.orchestra.run.smithi052.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.120145.log.gz 2024-05-04T21:22:42.147 INFO:teuthology.orchestra.run.smithi052.stderr: 84.3% 90.3% -- replaced with /var/log/ceph/ceph-client.admin.394912.log.gz -- replaced with /var/log/ceph/ceph-client.admin.419066.log.gz 2024-05-04T21:22:42.147 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-04T21:22:42.148 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.394497.log.gz 2024-05-04T21:22:42.149 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.216905.log.gz 2024-05-04T21:22:42.162 INFO:teuthology.orchestra.run.smithi052.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.425609.log.gz 2024-05-04T21:22:42.171 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.221391.log.gz 2024-05-04T21:22:42.171 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.221802.log.gz 2024-05-04T21:22:42.172 INFO:teuthology.orchestra.run.smithi052.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.197564.log.gz 2024-05-04T21:22:42.173 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.166792.log.gz 2024-05-04T21:22:42.173 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.171069.log.gz 2024-05-04T21:22:42.174 INFO:teuthology.orchestra.run.smithi052.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.172693.log.gz 2024-05-04T21:22:42.174 INFO:teuthology.orchestra.run.smithi052.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.228462.log.gz 2024-05-04T21:22:42.174 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.145445.log.gz 2024-05-04T21:22:42.175 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.173108.log.gz 2024-05-04T21:22:42.176 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36221.log.gz 2024-05-04T21:22:42.178 INFO:teuthology.orchestra.run.smithi052.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.0.37511.log.gz 2024-05-04T21:22:42.191 INFO:teuthology.orchestra.run.smithi052.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.298202.log.gz 2024-05-04T21:22:42.196 INFO:teuthology.orchestra.run.smithi052.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.105022.log.gz 2024-05-04T21:22:42.197 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35903.log.gz 2024-05-04T21:22:42.214 INFO:teuthology.orchestra.run.smithi052.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.312693.log.gz 2024-05-04T21:22:42.223 INFO:teuthology.orchestra.run.smithi052.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.242774.log.gz 2024-05-04T21:22:42.234 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.430410.log.gz 2024-05-04T21:22:42.237 INFO:teuthology.orchestra.run.smithi052.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.359036.log.gz 2024-05-04T21:22:42.253 INFO:teuthology.orchestra.run.smithi052.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.451291.log.gz 2024-05-04T21:22:42.265 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.473982.log.gz 2024-05-04T21:22:42.268 INFO:teuthology.orchestra.run.smithi052.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.183867.log.gz 2024-05-04T21:22:42.269 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.423945.log.gz 2024-05-04T21:22:42.269 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.426529.log.gz 2024-05-04T21:22:42.269 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.426935.log.gz 2024-05-04T21:22:42.270 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.184872.log.gz 2024-05-04T21:22:42.270 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.220993.log.gz 2024-05-04T21:22:42.293 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.438461.log.gz 2024-05-04T21:22:42.293 INFO:teuthology.orchestra.run.smithi052.stderr: 79.9% -- replaced with /var/log/ceph/ceph-client.admin.307222.log.gz 2024-05-04T21:22:42.295 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.330700.log.gz 2024-05-04T21:22:42.296 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.308057.log.gz 2024-05-04T21:22:42.297 INFO:teuthology.orchestra.run.smithi052.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.96805.log.gz 2024-05-04T21:22:42.297 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.316653.log.gz 2024-05-04T21:22:42.301 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.428711.log.gz 2024-05-04T21:22:42.304 INFO:teuthology.orchestra.run.smithi052.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.166366.log.gz 2024-05-04T21:22:42.304 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.108037.log.gz 2024-05-04T21:22:42.305 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.120154.log.gz 2024-05-04T21:22:42.307 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.198384.log.gz 2024-05-04T21:22:42.308 INFO:teuthology.orchestra.run.smithi052.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.159014.log.gz 2024-05-04T21:22:42.308 INFO:teuthology.orchestra.run.smithi052.stderr: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.158092.log.gz 2024-05-04T21:22:42.310 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.162878.log.gz 2024-05-04T21:22:42.310 INFO:teuthology.orchestra.run.smithi052.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.154384.log.gz 2024-05-04T21:22:42.311 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35212.log.gz 2024-05-04T21:22:42.321 INFO:teuthology.orchestra.run.smithi052.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.283679.log.gz 2024-05-04T21:22:42.326 INFO:teuthology.orchestra.run.smithi052.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34846.log.gz 2024-05-04T21:22:42.330 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.106715.log.gz 2024-05-04T21:22:42.353 INFO:teuthology.orchestra.run.smithi052.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35563.log.gz 2024-05-04T21:22:42.353 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.122728.log.gz 2024-05-04T21:22:42.353 INFO:teuthology.orchestra.run.smithi052.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33706.log.gz 2024-05-04T21:22:42.354 INFO:teuthology.orchestra.run.smithi052.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.153476.log.gz 2024-05-04T21:22:42.356 INFO:teuthology.orchestra.run.smithi052.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.295326.log.gz 2024-05-04T21:22:42.360 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36838.log.gz 2024-05-04T21:22:42.365 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.380490.log.gz 2024-05-04T21:22:42.366 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.412359.log.gz 2024-05-04T21:22:42.367 INFO:teuthology.orchestra.run.smithi052.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.344164.log.gz 2024-05-04T21:22:42.368 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.449801.log.gz 2024-05-04T21:22:42.368 INFO:teuthology.orchestra.run.smithi052.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.487304.log.gz 2024-05-04T21:22:42.370 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.424795.log.gz 2024-05-04T21:22:42.370 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.374859.log.gz 2024-05-04T21:22:42.370 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.479985.log.gz 2024-05-04T21:22:42.371 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.423539.log.gz 2024-05-04T21:22:42.381 INFO:teuthology.orchestra.run.smithi052.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.356952.log.gz 2024-05-04T21:22:42.388 INFO:teuthology.orchestra.run.smithi052.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.421901.log.gz 2024-05-04T21:22:42.389 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.454546.log.gz 2024-05-04T21:22:42.392 INFO:teuthology.orchestra.run.smithi052.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.188567.log.gz 2024-05-04T21:22:42.392 INFO:teuthology.orchestra.run.smithi052.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.162887.log.gz 2024-05-04T21:22:42.393 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.187764.log.gz 2024-05-04T21:22:42.405 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.361646.log.gz 2024-05-04T21:22:42.424 INFO:teuthology.orchestra.run.smithi052.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.202099.log.gz 2024-05-04T21:22:42.430 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.193892.log.gz 2024-05-04T21:22:42.430 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.194289.log.gz 2024-05-04T21:22:42.435 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.171890.log.gz 2024-05-04T21:22:42.435 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.228065.log.gz 2024-05-04T21:22:42.439 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.228471.log.gz 2024-05-04T21:22:42.446 INFO:teuthology.orchestra.run.smithi052.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36377.log.gz 2024-05-04T21:22:42.450 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.129103.log.gz 2024-05-04T21:22:42.457 INFO:teuthology.orchestra.run.smithi052.stderr: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.190702.log.gz 2024-05-04T21:22:42.472 INFO:teuthology.orchestra.run.smithi052.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.109297.log.gz 2024-05-04T21:22:42.480 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.320837.log.gz 2024-05-04T21:22:42.483 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.167236.log.gz 2024-05-04T21:22:42.483 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.411878.log.gz 2024-05-04T21:22:42.484 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.342881.log.gz 2024-05-04T21:22:42.484 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.467217.log.gz 2024-05-04T21:22:42.486 INFO:teuthology.orchestra.run.smithi052.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.373623.log.gz 2024-05-04T21:22:42.490 INFO:teuthology.orchestra.run.smithi052.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.159458.log.gz 2024-05-04T21:22:42.491 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.363297.log.gz 2024-05-04T21:22:42.496 INFO:teuthology.orchestra.run.smithi052.stderr: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.440080.log.gz 2024-05-04T21:22:42.498 INFO:teuthology.orchestra.run.smithi052.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.438452.log.gz 2024-05-04T21:22:42.499 INFO:teuthology.orchestra.run.smithi052.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.177169.log.gz 2024-05-04T21:22:42.501 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.181425.log.gz 2024-05-04T21:22:42.510 INFO:teuthology.orchestra.run.smithi052.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.80884.log.gz 2024-05-04T21:22:42.515 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.399218.log.gz 2024-05-04T21:22:42.516 INFO:teuthology.orchestra.run.smithi052.stderr: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.402958.log.gz 2024-05-04T21:22:42.520 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.175956.log.gz 2024-05-04T21:22:42.526 INFO:teuthology.orchestra.run.smithi052.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.79078.log.gz 2024-05-04T21:22:42.526 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.407630.log.gz 2024-05-04T21:22:42.527 INFO:teuthology.orchestra.run.smithi052.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.213150.log.gz 2024-05-04T21:22:42.527 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.416837.log.gz 2024-05-04T21:22:42.548 INFO:teuthology.orchestra.run.smithi052.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.482556.log.gz 2024-05-04T21:22:42.554 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.205903.log.gz 2024-05-04T21:22:42.558 INFO:teuthology.orchestra.run.smithi052.stderr: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.126821.log.gz 2024-05-04T21:22:42.559 INFO:teuthology.orchestra.run.smithi052.stderr: 79.7% -- replaced with /var/log/ceph/ceph-client.admin.209171.log.gz 2024-05-04T21:22:42.559 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.226026.log.gz 2024-05-04T21:22:42.566 INFO:teuthology.orchestra.run.smithi052.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.128536.log.gz 2024-05-04T21:22:42.566 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.127327.log.gz 2024-05-04T21:22:42.574 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.476124.log.gz 2024-05-04T21:22:42.575 INFO:teuthology.orchestra.run.smithi052.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.477411.log.gz 2024-05-04T21:22:42.575 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33916.log.gz 2024-05-04T21:22:42.576 INFO:teuthology.orchestra.run.smithi052.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.109708.log.gz 2024-05-04T21:22:42.576 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.110948.log.gz 2024-05-04T21:22:42.580 INFO:teuthology.orchestra.run.smithi052.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.324727.log.gz 2024-05-04T21:22:42.581 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.210848.log.gz 2024-05-04T21:22:42.602 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.243186.log.gz 2024-05-04T21:22:42.605 INFO:teuthology.orchestra.run.smithi052.stderr: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.474826.log.gz 2024-05-04T21:22:42.605 INFO:teuthology.orchestra.run.smithi052.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.324736.log.gz 2024-05-04T21:22:42.605 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.367460.log.gz 2024-05-04T21:22:42.606 INFO:teuthology.orchestra.run.smithi052.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.351955.log.gz 2024-05-04T21:22:42.608 INFO:teuthology.orchestra.run.smithi052.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.485574.log.gz 2024-05-04T21:22:42.608 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.374034.log.gz 2024-05-04T21:22:42.609 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.473167.log.gz 2024-05-04T21:22:42.614 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.396948.log.gz 2024-05-04T21:22:42.618 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37139.log.gz 2024-05-04T21:22:42.630 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.438867.log.gz 2024-05-04T21:22:42.630 INFO:teuthology.orchestra.run.smithi052.stderr: 79.6% -- replaced with /var/log/ceph/ceph-client.admin.446769.log.gz 2024-05-04T21:22:42.636 INFO:teuthology.orchestra.run.smithi052.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.400894.log.gz 2024-05-04T21:22:42.636 INFO:teuthology.orchestra.run.smithi052.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.122297.log.gz 2024-05-04T21:22:42.636 INFO:teuthology.orchestra.run.smithi052.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.425202.log.gz 2024-05-04T21:22:42.638 INFO:teuthology.orchestra.run.smithi052.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.345015.log.gz 2024-05-04T21:22:42.640 INFO:teuthology.orchestra.run.smithi052.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.293626.log.gz 2024-05-04T21:22:42.640 INFO:teuthology.orchestra.run.smithi052.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.379574.log.gz 2024-05-04T21:22:42.643 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.403373.log.gz 2024-05-04T21:22:42.661 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.369902.log.gz 2024-05-04T21:22:42.661 INFO:teuthology.orchestra.run.smithi052.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.228056.log.gz 2024-05-04T21:22:42.662 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.226847.log.gz 2024-05-04T21:22:42.670 INFO:teuthology.orchestra.run.smithi052.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34950.log.gz 2024-05-04T21:22:42.676 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.212540.log.gz 2024-05-04T21:22:42.680 INFO:teuthology.orchestra.run.smithi052.stderr: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.200460.log.gz 2024-05-04T21:22:42.684 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.155796.log.gz 2024-05-04T21:22:42.686 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% 90.3% -- replaced with /var/log/ceph/ceph-client.admin.33689.log.gz -- replaced with /var/log/ceph/ceph-client.admin.147500.log.gz 2024-05-04T21:22:42.686 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-04T21:22:42.687 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.86661.log.gz 2024-05-04T21:22:42.708 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.118877.log.gz 2024-05-04T21:22:42.708 INFO:teuthology.orchestra.run.smithi052.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.484327.log.gz 2024-05-04T21:22:42.715 INFO:teuthology.orchestra.run.smithi052.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.440922.log.gz 2024-05-04T21:22:42.716 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.454147.log.gz 2024-05-04T21:22:42.717 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.439681.log.gz 2024-05-04T21:22:42.720 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.372398.log.gz 2024-05-04T21:22:42.722 INFO:teuthology.orchestra.run.smithi052.stderr: 79.7% -- replaced with /var/log/ceph/ceph-client.admin.187349.log.gz 2024-05-04T21:22:42.723 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.147905.log.gz 2024-05-04T21:22:42.724 INFO:teuthology.orchestra.run.smithi052.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.180177.log.gz 2024-05-04T21:22:42.725 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.400488.log.gz 2024-05-04T21:22:42.725 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.305190.log.gz 2024-05-04T21:22:42.727 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.411408.log.gz 2024-05-04T21:22:42.731 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.315807.log.gz 2024-05-04T21:22:42.732 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.469407.log.gz 2024-05-04T21:22:42.738 INFO:teuthology.orchestra.run.smithi052.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.178450.log.gz 2024-05-04T21:22:42.742 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.406349.log.gz 2024-05-04T21:22:42.745 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.232604.log.gz 2024-05-04T21:22:42.745 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.148765.log.gz 2024-05-04T21:22:42.746 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.161168.log.gz 2024-05-04T21:22:42.747 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.196760.log.gz 2024-05-04T21:22:42.749 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.226035.log.gz 2024-05-04T21:22:42.755 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.294108.log.gz 2024-05-04T21:22:42.755 INFO:teuthology.orchestra.run.smithi052.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35428.log.gz 2024-05-04T21:22:42.756 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35506.log.gz 2024-05-04T21:22:42.756 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35877.log.gz 2024-05-04T21:22:42.760 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36663.log.gz 2024-05-04T21:22:42.760 INFO:teuthology.orchestra.run.smithi052.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.121833.log.gz 2024-05-04T21:22:42.763 INFO:teuthology.orchestra.run.smithi052.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.211651.log.gz 2024-05-04T21:22:42.764 INFO:teuthology.orchestra.run.smithi052.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.476991.log.gz 2024-05-04T21:22:42.765 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.317063.log.gz 2024-05-04T21:22:42.765 INFO:teuthology.orchestra.run.smithi052.stderr: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.431213.log.gz 2024-05-04T21:22:42.765 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.322045.log.gz 2024-05-04T21:22:42.769 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.364572.log.gz 2024-05-04T21:22:42.769 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.365380.log.gz 2024-05-04T21:22:42.770 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.148339.log.gz 2024-05-04T21:22:42.771 INFO:teuthology.orchestra.run.smithi052.stderr: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.343342.log.gz 2024-05-04T21:22:42.771 INFO:teuthology.orchestra.run.smithi052.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.284099.log.gz 2024-05-04T21:22:42.773 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.108436.log.gz 2024-05-04T21:22:42.778 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.474429.log.gz 2024-05-04T21:22:42.778 INFO:teuthology.orchestra.run.smithi052.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.186081.log.gz 2024-05-04T21:22:42.779 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.223446.log.gz 2024-05-04T21:22:42.781 INFO:teuthology.orchestra.run.smithi052.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.347123.log.gz 2024-05-04T21:22:42.784 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.349470.log.gz 2024-05-04T21:22:42.787 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.104216.log.gz 2024-05-04T21:22:42.803 INFO:teuthology.orchestra.run.smithi052.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.476562.log.gz 2024-05-04T21:22:42.806 INFO:teuthology.orchestra.run.smithi052.stderr: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.233409.log.gz 2024-05-04T21:22:42.811 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.230510.log.gz 2024-05-04T21:22:42.812 INFO:teuthology.orchestra.run.smithi052.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.481699.log.gz 2024-05-04T21:22:42.876 INFO:teuthology.orchestra.run.smithi052.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.104207.log.gz 2024-05-04T21:22:42.877 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.103807.log.gz 2024-05-04T21:22:42.917 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.350730.log.gz 2024-05-04T21:22:42.923 INFO:teuthology.orchestra.run.smithi111.stderr: 94.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-04T21:22:42.932 INFO:teuthology.orchestra.run.smithi052.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44704.log.gz 2024-05-04T21:22:42.933 INFO:teuthology.orchestra.run.smithi052.stderr: 77.7% -- replaced with /var/log/ceph/ceph-client.admin.396121.log.gz 2024-05-04T21:22:42.935 INFO:teuthology.orchestra.run.smithi052.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.241509.log.gz 2024-05-04T21:22:42.935 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.485168.log.gz 2024-05-04T21:22:42.942 INFO:teuthology.orchestra.run.smithi052.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.364158.log.gz 2024-05-04T21:22:42.943 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.459420.log.gz 2024-05-04T21:22:42.944 INFO:teuthology.orchestra.run.smithi052.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.479994.log.gz 2024-05-04T21:22:42.945 INFO:teuthology.orchestra.run.smithi052.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.383781.log.gz 2024-05-04T21:22:42.946 INFO:teuthology.orchestra.run.smithi052.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.316204.log.gz 2024-05-04T21:22:42.949 INFO:teuthology.orchestra.run.smithi052.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32781.log.gz 2024-05-04T21:22:42.951 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.175131.log.gz 2024-05-04T21:22:42.953 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34124.log.gz 2024-05-04T21:22:42.954 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34202.log.gz 2024-05-04T21:22:42.955 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.96395.log.gz 2024-05-04T21:22:42.958 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.98512.log.gz 2024-05-04T21:22:42.961 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.101775.log.gz 2024-05-04T21:22:42.966 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.408081.log.gz 2024-05-04T21:22:42.976 INFO:teuthology.orchestra.run.smithi052.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.396957.log.gz 2024-05-04T21:22:42.996 INFO:teuthology.orchestra.run.smithi052.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.362459.log.gz 2024-05-04T21:22:43.006 INFO:teuthology.orchestra.run.smithi052.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36034.log.gz 2024-05-04T21:22:43.006 INFO:teuthology.orchestra.run.smithi052.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33500.log.gz 2024-05-04T21:22:43.011 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37327.log.gz 2024-05-04T21:22:43.011 INFO:teuthology.orchestra.run.smithi052.stderr: 83.5% -- replaced with /var/log/ceph/ceph-client.admin.98913.log.gz 2024-05-04T21:22:43.034 INFO:teuthology.orchestra.run.smithi052.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.202539.log.gz 2024-05-04T21:22:43.035 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36786.log.gz 2024-05-04T21:22:43.039 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.411000.log.gz 2024-05-04T21:22:43.039 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.282057.log.gz 2024-05-04T21:22:43.041 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.286549.log.gz 2024-05-04T21:22:43.043 INFO:teuthology.orchestra.run.smithi052.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.313172.log.gz 2024-05-04T21:22:43.043 INFO:teuthology.orchestra.run.smithi052.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.359336.log.gz 2024-05-04T21:22:43.046 INFO:teuthology.orchestra.run.smithi052.stderr: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.372796.log.gz 2024-05-04T21:22:43.047 INFO:teuthology.orchestra.run.smithi052.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.485583.log.gz 2024-05-04T21:22:43.049 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.359027.log.gz 2024-05-04T21:22:43.051 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.183456.log.gz 2024-05-04T21:22:43.052 INFO:teuthology.orchestra.run.smithi052.stderr: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.308048.log.gz 2024-05-04T21:22:43.054 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.456982.log.gz 2024-05-04T21:22:43.063 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.95169.log.gz 2024-05-04T21:22:43.064 INFO:teuthology.orchestra.run.smithi052.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.169843.log.gz 2024-05-04T21:22:43.064 INFO:teuthology.orchestra.run.smithi052.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.196751.log.gz 2024-05-04T21:22:43.068 INFO:teuthology.orchestra.run.smithi052.stderr: 80.0% -- replaced with /var/log/ceph/ceph-client.admin.280825.log.gz 2024-05-04T21:22:43.069 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.239299.log.gz 2024-05-04T21:22:43.069 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.227659.log.gz 2024-05-04T21:22:43.070 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.385266.log.gz 2024-05-04T21:22:43.074 INFO:teuthology.orchestra.run.smithi111.stderr: 94.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-04T21:22:43.078 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.204684.log.gz 2024-05-04T21:22:43.078 INFO:teuthology.orchestra.run.smithi052.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36169.log.gz 2024-05-04T21:22:43.081 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.215201.log.gz 2024-05-04T21:22:43.109 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35480.log.gz 2024-05-04T21:22:43.110 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36008.log.gz 2024-05-04T21:22:43.159 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.321648.log.gz 2024-05-04T21:22:43.162 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.312232.log.gz 2024-05-04T21:22:43.162 INFO:teuthology.orchestra.run.smithi052.stderr: 83.5% -- replaced with /var/log/ceph/ceph-client.admin.314985.log.gz 2024-05-04T21:22:43.163 INFO:teuthology.orchestra.run.smithi052.stderr: 83.1% -- replaced with /var/log/ceph/ceph-client.admin.317469.log.gz 2024-05-04T21:22:43.164 INFO:teuthology.orchestra.run.smithi052.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.322931.log.gz 2024-05-04T21:22:43.164 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.317072.log.gz 2024-05-04T21:22:43.168 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.432035.log.gz 2024-05-04T21:22:43.169 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.191599.log.gz 2024-05-04T21:22:43.170 INFO:teuthology.orchestra.run.smithi052.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.433263.log.gz 2024-05-04T21:22:43.172 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.433272.log.gz 2024-05-04T21:22:43.173 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.364563.log.gz 2024-05-04T21:22:43.175 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.364983.log.gz 2024-05-04T21:22:43.176 INFO:teuthology.orchestra.run.smithi052.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.167670.log.gz 2024-05-04T21:22:43.177 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.243592.log.gz 2024-05-04T21:22:43.179 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.366238.log.gz 2024-05-04T21:22:43.180 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.349899.log.gz 2024-05-04T21:22:43.181 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.373632.log.gz 2024-05-04T21:22:43.183 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.176771.log.gz 2024-05-04T21:22:43.183 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.180605.log.gz 2024-05-04T21:22:43.187 INFO:teuthology.orchestra.run.smithi052.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.161612.log.gz 2024-05-04T21:22:43.188 INFO:teuthology.orchestra.run.smithi052.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.221811.log.gz 2024-05-04T21:22:43.189 INFO:teuthology.orchestra.run.smithi052.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.378137.log.gz 2024-05-04T21:22:43.197 INFO:teuthology.orchestra.run.smithi052.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.170253.log.gz 2024-05-04T21:22:43.197 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.197573.log.gz 2024-05-04T21:22:43.198 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.280834.log.gz 2024-05-04T21:22:43.200 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.178894.log.gz 2024-05-04T21:22:43.212 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.163756.log.gz 2024-05-04T21:22:43.228 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.418666.log.gz 2024-05-04T21:22:43.247 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.205505.log.gz 2024-05-04T21:22:43.258 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.314118.log.gz 2024-05-04T21:22:43.261 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.191162.log.gz 2024-05-04T21:22:43.261 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37223.log.gz 2024-05-04T21:22:43.267 INFO:teuthology.orchestra.run.smithi052.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.364149.log.gz 2024-05-04T21:22:43.278 INFO:teuthology.orchestra.run.smithi052.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.286540.log.gz 2024-05-04T21:22:43.290 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.355222.log.gz 2024-05-04T21:22:43.291 INFO:teuthology.orchestra.run.smithi052.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.437170.log.gz 2024-05-04T21:22:43.292 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.320431.log.gz 2024-05-04T21:22:43.294 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.459429.log.gz 2024-05-04T21:22:43.298 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.368272.log.gz 2024-05-04T21:22:43.299 INFO:teuthology.orchestra.run.smithi052.stderr: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.384186.log.gz 2024-05-04T21:22:43.301 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.409696.log.gz 2024-05-04T21:22:43.306 INFO:teuthology.orchestra.run.smithi052.stderr: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.482118.log.gz 2024-05-04T21:22:43.307 INFO:teuthology.orchestra.run.smithi052.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.471877.log.gz 2024-05-04T21:22:43.308 INFO:teuthology.orchestra.run.smithi052.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.126830.log.gz 2024-05-04T21:22:43.308 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.363708.log.gz 2024-05-04T21:22:43.319 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.146664.log.gz 2024-05-04T21:22:43.320 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.233011.log.gz 2024-05-04T21:22:43.320 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.479136.log.gz 2024-05-04T21:22:43.324 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.158498.log.gz 2024-05-04T21:22:43.324 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.165487.log.gz 2024-05-04T21:22:43.324 INFO:teuthology.orchestra.run.smithi052.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.171881.log.gz 2024-05-04T21:22:43.328 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.164191.log.gz 2024-05-04T21:22:43.331 INFO:teuthology.orchestra.run.smithi052.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.240564.log.gz 2024-05-04T21:22:43.399 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.103391.log.gz 2024-05-04T21:22:43.405 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.434977.log.gz 2024-05-04T21:22:43.405 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.315400.log.gz 2024-05-04T21:22:43.415 INFO:teuthology.orchestra.run.smithi052.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.295741.log.gz 2024-05-04T21:22:43.419 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.484743.log.gz 2024-05-04T21:22:43.420 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.383385.log.gz 2024-05-04T21:22:43.430 INFO:teuthology.orchestra.run.smithi052.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.185269.log.gz 2024-05-04T21:22:43.431 INFO:teuthology.orchestra.run.smithi052.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.192927.log.gz 2024-05-04T21:22:43.503 INFO:teuthology.orchestra.run.smithi052.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34684.log.gz 2024-05-04T21:22:43.505 INFO:teuthology.orchestra.run.smithi052.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.464144.log.gz 2024-05-04T21:22:43.509 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.478277.log.gz 2024-05-04T21:22:43.511 INFO:teuthology.orchestra.run.smithi052.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.480832.log.gz 2024-05-04T21:22:43.514 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.421438.log.gz 2024-05-04T21:22:43.519 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.289411.log.gz 2024-05-04T21:22:43.523 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.437644.log.gz 2024-05-04T21:22:43.527 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.102172.log.gz 2024-05-04T21:22:43.531 INFO:teuthology.orchestra.run.smithi052.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.459836.log.gz 2024-05-04T21:22:43.534 INFO:teuthology.orchestra.run.smithi052.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.212111.log.gz 2024-05-04T21:22:43.540 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.294117.log.gz 2024-05-04T21:22:43.540 INFO:teuthology.orchestra.run.smithi052.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.296172.log.gz 2024-05-04T21:22:43.544 INFO:teuthology.orchestra.run.smithi052.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.308458.log.gz 2024-05-04T21:22:43.547 INFO:teuthology.orchestra.run.smithi052.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.108890.log.gz 2024-05-04T21:22:43.556 INFO:teuthology.orchestra.run.smithi052.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.209610.log.gz 2024-05-04T21:22:43.571 INFO:teuthology.orchestra.run.smithi052.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.404598.log.gz 2024-05-04T21:22:43.585 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35297.log.gz 2024-05-04T21:22:43.588 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33691.log.gz 2024-05-04T21:22:43.621 INFO:teuthology.orchestra.run.smithi052.stderr: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.432026.log.gz 2024-05-04T21:22:43.621 INFO:teuthology.orchestra.run.smithi052.stderr: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.437635.log.gz 2024-05-04T21:22:43.622 INFO:teuthology.orchestra.run.smithi052.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.367857.log.gz 2024-05-04T21:22:43.622 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.414181.log.gz 2024-05-04T21:22:43.622 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.429117.log.gz 2024-05-04T21:22:43.622 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.367866.log.gz 2024-05-04T21:22:43.639 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.349064.log.gz 2024-05-04T21:22:43.645 INFO:teuthology.orchestra.run.smithi052.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.485159.log.gz 2024-05-04T21:22:43.681 INFO:teuthology.orchestra.run.smithi052.stderr: 78.5% -- replaced with /var/log/ceph/ceph-client.admin.469398.log.gz 2024-05-04T21:22:43.682 INFO:teuthology.orchestra.run.smithi052.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.468451.log.gz 2024-05-04T21:22:43.684 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.210437.log.gz 2024-05-04T21:22:43.684 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34898.log.gz 2024-05-04T21:22:43.685 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.164182.log.gz 2024-05-04T21:22:43.687 INFO:teuthology.orchestra.run.smithi052.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.171475.log.gz 2024-05-04T21:22:43.690 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.413784.log.gz 2024-05-04T21:22:43.692 INFO:teuthology.orchestra.run.smithi052.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36299.log.gz 2024-05-04T21:22:43.695 INFO:teuthology.orchestra.run.smithi052.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.372805.log.gz 2024-05-04T21:22:43.697 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.113054.log.gz 2024-05-04T21:22:43.701 INFO:teuthology.orchestra.run.smithi052.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.406791.log.gz 2024-05-04T21:22:43.708 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.151762.log.gz 2024-05-04T21:22:43.712 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.306825.log.gz 2024-05-04T21:22:43.712 INFO:teuthology.orchestra.run.smithi052.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.94356.log.gz 2024-05-04T21:22:43.715 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.95990.log.gz 2024-05-04T21:22:43.728 INFO:teuthology.orchestra.run.smithi052.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.468028.log.gz 2024-05-04T21:22:43.750 INFO:teuthology.orchestra.run.smithi052.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33052.log.gz 2024-05-04T21:22:43.751 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33370.log.gz 2024-05-04T21:22:43.781 INFO:teuthology.orchestra.run.smithi052.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.429108.log.gz 2024-05-04T21:22:43.783 INFO:teuthology.orchestra.run.smithi052.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.435373.log.gz 2024-05-04T21:22:43.783 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.431628.log.gz 2024-05-04T21:22:43.802 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.359433.log.gz 2024-05-04T21:22:43.809 INFO:teuthology.orchestra.run.smithi052.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.426926.log.gz 2024-05-04T21:22:43.810 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.202965.log.gz 2024-05-04T21:22:43.821 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.359287.log.gz 2024-05-04T21:22:43.858 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.287357.log.gz 2024-05-04T21:22:43.864 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.104623.log.gz 2024-05-04T21:22:43.866 INFO:teuthology.orchestra.run.smithi052.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.169413.log.gz 2024-05-04T21:22:43.866 INFO:teuthology.orchestra.run.smithi052.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.408478.log.gz 2024-05-04T21:22:43.867 INFO:teuthology.orchestra.run.smithi052.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.408884.log.gz 2024-05-04T21:22:43.868 INFO:teuthology.orchestra.run.smithi052.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.156834.log.gz 2024-05-04T21:22:43.869 INFO:teuthology.orchestra.run.smithi052.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36866.log.gz 2024-05-04T21:22:43.869 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36918.log.gz 2024-05-04T21:22:43.871 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.398265.log.gz 2024-05-04T21:22:43.872 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.399615.log.gz 2024-05-04T21:22:43.873 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.164617.log.gz 2024-05-04T21:22:43.874 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.226838.log.gz 2024-05-04T21:22:43.875 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.453326.log.gz 2024-05-04T21:22:43.875 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33240.log.gz 2024-05-04T21:22:43.876 INFO:teuthology.orchestra.run.smithi052.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.422716.log.gz 2024-05-04T21:22:43.877 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36273.log.gz 2024-05-04T21:22:43.886 INFO:teuthology.orchestra.run.smithi052.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.376489.log.gz 2024-05-04T21:22:43.887 INFO:teuthology.orchestra.run.smithi052.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.342872.log.gz 2024-05-04T21:22:43.895 INFO:teuthology.orchestra.run.smithi052.stderr: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.304774.log.gz 2024-05-04T21:22:43.900 INFO:teuthology.orchestra.run.smithi052.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.428702.log.gz 2024-05-04T21:22:43.905 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.332131.log.gz 2024-05-04T21:22:43.915 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.160307.log.gz 2024-05-04T21:22:43.916 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.127733.log.gz 2024-05-04T21:22:43.917 INFO:teuthology.orchestra.run.smithi052.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.241951.log.gz 2024-05-04T21:22:43.922 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.362057.log.gz 2024-05-04T21:22:43.927 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.81302.log.gz 2024-05-04T21:22:43.927 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33422.log.gz 2024-05-04T21:22:43.927 INFO:teuthology.orchestra.run.smithi052.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.95575.log.gz 2024-05-04T21:22:43.945 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.105431.log.gz 2024-05-04T21:22:43.949 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.345873.log.gz 2024-05-04T21:22:44.002 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.457399.log.gz 2024-05-04T21:22:44.003 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.423133.log.gz 2024-05-04T21:22:44.007 INFO:teuthology.orchestra.run.smithi052.stderr: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.402552.log.gz 2024-05-04T21:22:44.010 INFO:teuthology.orchestra.run.smithi052.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.370312.log.gz 2024-05-04T21:22:44.011 INFO:teuthology.orchestra.run.smithi052.stderr: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.220573.log.gz 2024-05-04T21:22:44.014 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.489021.log.gz 2024-05-04T21:22:44.022 INFO:teuthology.orchestra.run.smithi052.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.157240.log.gz 2024-05-04T21:22:44.023 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.232155.log.gz 2024-05-04T21:22:44.024 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.454555.log.gz 2024-05-04T21:22:44.025 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.147084.log.gz 2024-05-04T21:22:44.027 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.415174.log.gz 2024-05-04T21:22:44.030 INFO:teuthology.orchestra.run.smithi052.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.229681.log.gz 2024-05-04T21:22:44.033 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.200062.log.gz 2024-05-04T21:22:44.033 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.345467.log.gz 2024-05-04T21:22:44.037 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.144633.log.gz 2024-05-04T21:22:44.041 INFO:teuthology.orchestra.run.smithi052.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.297797.log.gz 2024-05-04T21:22:44.076 INFO:teuthology.orchestra.run.smithi052.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.173911.log.gz 2024-05-04T21:22:44.077 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.173920.log.gz 2024-05-04T21:22:44.113 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35454.log.gz 2024-05-04T21:22:44.114 INFO:teuthology.orchestra.run.smithi052.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35589.log.gz 2024-05-04T21:22:44.114 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.361226.log.gz -- replaced with /var/log/ceph/ceph-client.admin.33266.log.gz 2024-05-04T21:22:44.114 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-04T21:22:44.115 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.468037.log.gz 2024-05-04T21:22:44.116 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.94771.log.gz 2024-05-04T21:22:44.116 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.350310.log.gz 2024-05-04T21:22:44.129 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.482547.log.gz 2024-05-04T21:22:44.137 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.192464.log.gz 2024-05-04T21:22:44.137 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.203790.log.gz 2024-05-04T21:22:44.139 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.395724.log.gz 2024-05-04T21:22:44.151 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36611.log.gz 2024-05-04T21:22:44.155 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.420624.log.gz 2024-05-04T21:22:44.156 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.413312.log.gz 2024-05-04T21:22:44.158 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.471061.log.gz 2024-05-04T21:22:44.166 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.224265.log.gz 2024-05-04T21:22:44.166 INFO:teuthology.orchestra.run.smithi052.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.233002.log.gz 2024-05-04T21:22:44.167 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.194298.log.gz 2024-05-04T21:22:44.167 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.145847.log.gz 2024-05-04T21:22:44.169 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.102578.log.gz 2024-05-04T21:22:44.169 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35002.log.gz 2024-05-04T21:22:44.180 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.377325.log.gz 2024-05-04T21:22:44.185 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.151346.log.gz 2024-05-04T21:22:44.259 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33483.log.gz 2024-05-04T21:22:44.262 INFO:teuthology.orchestra.run.smithi052.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.479127.log.gz 2024-05-04T21:22:44.263 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33495.log.gz 2024-05-04T21:22:44.263 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33690.log.gz 2024-05-04T21:22:44.263 INFO:teuthology.orchestra.run.smithi052.stderr: 78.8% -- replaced with /var/log/ceph/ceph-client.admin.312223.log.gz 2024-05-04T21:22:44.263 INFO:teuthology.orchestra.run.smithi052.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.462452.log.gz 2024-05-04T21:22:44.263 INFO:teuthology.orchestra.run.smithi052.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.105440.log.gz 2024-05-04T21:22:44.267 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34872.log.gz 2024-05-04T21:22:44.273 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.431619.log.gz 2024-05-04T21:22:44.273 INFO:teuthology.orchestra.run.smithi052.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.436184.log.gz 2024-05-04T21:22:44.274 INFO:teuthology.orchestra.run.smithi052.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.382539.log.gz 2024-05-04T21:22:44.277 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.353219.log.gz 2024-05-04T21:22:44.277 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.353625.log.gz 2024-05-04T21:22:44.278 INFO:teuthology.orchestra.run.smithi052.stderr: 78.8% -- replaced with /var/log/ceph/ceph-client.admin.477840.log.gz 2024-05-04T21:22:44.279 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.353228.log.gz 2024-05-04T21:22:44.279 INFO:teuthology.orchestra.run.smithi052.stderr: 78.8% -- replaced with /var/log/ceph/ceph-client.admin.465907.log.gz 2024-05-04T21:22:44.279 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.179762.log.gz 2024-05-04T21:22:44.281 INFO:teuthology.orchestra.run.smithi052.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.112651.log.gz 2024-05-04T21:22:44.289 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.476133.log.gz 2024-05-04T21:22:44.290 INFO:teuthology.orchestra.run.smithi052.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.233825.log.gz 2024-05-04T21:22:44.298 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.167227.log.gz 2024-05-04T21:22:44.302 INFO:teuthology.orchestra.run.smithi052.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.301445.log.gz 2024-05-04T21:22:44.302 INFO:teuthology.orchestra.run.smithi052.stderr: 94.6% -- replaced with /var/log/ceph/ceph-client.admin.480423.log.gz 2024-05-04T21:22:44.303 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36143.log.gz 2024-05-04T21:22:44.304 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.301454.log.gz 2024-05-04T21:22:44.306 INFO:teuthology.orchestra.run.smithi052.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.128843.log.gz 2024-05-04T21:22:44.309 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.150476.log.gz 2024-05-04T21:22:44.336 INFO:teuthology.orchestra.run.smithi052.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.97243.log.gz 2024-05-04T21:22:44.336 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.118050.log.gz 2024-05-04T21:22:44.379 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34768.log.gz 2024-05-04T21:22:44.380 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35323.log.gz 2024-05-04T21:22:44.405 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.314579.log.gz 2024-05-04T21:22:44.408 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.432439.log.gz 2024-05-04T21:22:44.435 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37048.log.gz 2024-05-04T21:22:44.436 INFO:teuthology.orchestra.run.smithi052.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37353.log.gz 2024-05-04T21:22:44.437 INFO:teuthology.orchestra.run.smithi052.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.37405.log.gz 2024-05-04T21:22:44.437 INFO:teuthology.orchestra.run.smithi052.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.234250.log.gz 2024-05-04T21:22:44.438 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.234690.log.gz 2024-05-04T21:22:44.441 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.458617.log.gz 2024-05-04T21:22:44.442 INFO:teuthology.orchestra.run.smithi052.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.402967.log.gz 2024-05-04T21:22:44.448 INFO:teuthology.orchestra.run.smithi052.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.160742.log.gz 2024-05-04T21:22:44.458 INFO:teuthology.orchestra.run.smithi052.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.286951.log.gz 2024-05-04T21:22:44.469 INFO:teuthology.orchestra.run.smithi052.stderr: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.465469.log.gz 2024-05-04T21:22:44.469 INFO:teuthology.orchestra.run.smithi052.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.200868.log.gz 2024-05-04T21:22:44.477 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33132.log.gz 2024-05-04T21:22:44.478 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36091.log.gz 2024-05-04T21:22:44.479 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.290639.log.gz 2024-05-04T21:22:44.484 INFO:teuthology.orchestra.run.smithi052.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.299417.log.gz 2024-05-04T21:22:44.485 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.298211.log.gz 2024-05-04T21:22:44.488 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.98105.log.gz 2024-05-04T21:22:44.496 INFO:teuthology.orchestra.run.smithi052.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.200053.log.gz 2024-05-04T21:22:44.561 INFO:teuthology.orchestra.run.smithi052.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.90587.log.gz 2024-05-04T21:22:44.561 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.91048.log.gz 2024-05-04T21:22:44.569 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.424342.log.gz 2024-05-04T21:22:44.579 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.326875.log.gz 2024-05-04T21:22:44.586 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.244002.log.gz 2024-05-04T21:22:44.589 INFO:teuthology.orchestra.run.smithi052.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.234259.log.gz 2024-05-04T21:22:44.590 INFO:teuthology.orchestra.run.smithi052.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.285722.log.gz 2024-05-04T21:22:44.591 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.403771.log.gz 2024-05-04T21:22:44.595 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.417318.log.gz 2024-05-04T21:22:44.602 INFO:teuthology.orchestra.run.smithi052.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.93090.log.gz 2024-05-04T21:22:44.604 INFO:teuthology.orchestra.run.smithi052.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.159859.log.gz 2024-05-04T21:22:44.605 INFO:teuthology.orchestra.run.smithi052.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.163312.log.gz 2024-05-04T21:22:44.605 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.146262.log.gz 2024-05-04T21:22:44.608 INFO:teuthology.orchestra.run.smithi052.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.169852.log.gz 2024-05-04T21:22:44.610 INFO:teuthology.orchestra.run.smithi052.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.300633.log.gz 2024-05-04T21:22:44.627 INFO:teuthology.orchestra.run.smithi052.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37191.log.gz 2024-05-04T21:22:44.679 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.119731.log.gz 2024-05-04T21:22:44.679 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.102181.log.gz 2024-05-04T21:22:44.679 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.311785.log.gz 2024-05-04T21:22:44.684 INFO:teuthology.orchestra.run.smithi052.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.324270.log.gz 2024-05-04T21:22:44.685 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.291041.log.gz 2024-05-04T21:22:44.686 INFO:teuthology.orchestra.run.smithi052.stderr: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.413303.log.gz 2024-05-04T21:22:44.688 INFO:teuthology.orchestra.run.smithi052.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.355628.log.gz 2024-05-04T21:22:44.689 INFO:teuthology.orchestra.run.smithi052.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.352776.log.gz 2024-05-04T21:22:44.689 INFO:teuthology.orchestra.run.smithi052.stderr: 83.5% -- replaced with /var/log/ceph/ceph-client.admin.181822.log.gz 2024-05-04T21:22:44.689 INFO:teuthology.orchestra.run.smithi052.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.283688.log.gz 2024-05-04T21:22:44.695 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.420209.log.gz 2024-05-04T21:22:44.698 INFO:teuthology.orchestra.run.smithi052.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.326445.log.gz 2024-05-04T21:22:44.699 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.187755.log.gz 2024-05-04T21:22:44.703 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.218951.log.gz 2024-05-04T21:22:44.704 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.205912.log.gz 2024-05-04T21:22:44.708 INFO:teuthology.orchestra.run.smithi052.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.396536.log.gz 2024-05-04T21:22:44.710 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.464135.log.gz 2024-05-04T21:22:44.715 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.294523.log.gz 2024-05-04T21:22:44.720 INFO:teuthology.orchestra.run.smithi052.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33186.log.gz 2024-05-04T21:22:44.848 INFO:teuthology.orchestra.run.smithi052.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.95584.log.gz 2024-05-04T21:22:44.860 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.101766.log.gz 2024-05-04T21:22:44.862 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.104614.log.gz 2024-05-04T21:22:44.863 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.103400.log.gz 2024-05-04T21:22:44.865 INFO:teuthology.orchestra.run.smithi052.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.121369.log.gz 2024-05-04T21:22:44.871 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.374454.log.gz 2024-05-04T21:22:44.874 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.319195.log.gz 2024-05-04T21:22:44.878 INFO:teuthology.orchestra.run.smithi052.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.366639.log.gz 2024-05-04T21:22:44.878 INFO:teuthology.orchestra.run.smithi052.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.364974.log.gz 2024-05-04T21:22:44.884 INFO:teuthology.orchestra.run.smithi052.stderr: 83.7% -- replaced with /var/log/ceph/ceph-client.admin.409290.log.gz 2024-05-04T21:22:44.884 INFO:teuthology.orchestra.run.smithi052.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.282862.log.gz 2024-05-04T21:22:44.891 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.472348.log.gz 2024-05-04T21:22:44.893 INFO:teuthology.orchestra.run.smithi052.stderr: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.305588.log.gz 2024-05-04T21:22:44.894 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.202974.log.gz 2024-05-04T21:22:44.898 INFO:teuthology.orchestra.run.smithi052.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.420615.log.gz 2024-05-04T21:22:44.899 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.399624.log.gz 2024-05-04T21:22:44.901 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.406340.log.gz 2024-05-04T21:22:44.902 INFO:teuthology.orchestra.run.smithi052.stderr: 79.9% -- replaced with /var/log/ceph/ceph-client.admin.417779.log.gz 2024-05-04T21:22:44.904 INFO:teuthology.orchestra.run.smithi052.stderr: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.302270.log.gz 2024-05-04T21:22:44.905 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.233418.log.gz 2024-05-04T21:22:44.906 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.475703.log.gz 2024-05-04T21:22:44.907 INFO:teuthology.orchestra.run.smithi052.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.328820.log.gz 2024-05-04T21:22:44.910 INFO:teuthology.orchestra.run.smithi052.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.371547.log.gz 2024-05-04T21:22:44.918 INFO:teuthology.orchestra.run.smithi052.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.165931.log.gz 2024-05-04T21:22:44.927 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35158.log.gz 2024-05-04T21:22:44.938 INFO:teuthology.orchestra.run.smithi052.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.213576.log.gz 2024-05-04T21:22:44.938 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.197979.log.gz 2024-05-04T21:22:44.941 INFO:teuthology.orchestra.run.smithi052.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.149234.log.gz 2024-05-04T21:22:44.958 INFO:teuthology.orchestra.run.smithi052.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.365801.log.gz 2024-05-04T21:22:45.016 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36060.log.gz 2024-05-04T21:22:45.016 INFO:teuthology.orchestra.run.smithi052.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33502.log.gz 2024-05-04T21:22:45.047 INFO:teuthology.orchestra.run.smithi052.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.243993.log.gz 2024-05-04T21:22:45.048 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.432860.log.gz 2024-05-04T21:22:45.049 INFO:teuthology.orchestra.run.smithi052.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.283282.log.gz 90.3% 2024-05-04T21:22:45.049 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.285320.log.gz 2024-05-04T21:22:45.187 INFO:teuthology.orchestra.run.smithi052.stderr: 77.6% 91.8% -- replaced with /var/log/ceph/ceph-client.admin.454952.log.gz -- replaced with /var/log/ceph/ceph-client.admin.203380.log.gz 2024-05-04T21:22:45.187 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-04T21:22:45.188 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.398746.log.gz 2024-05-04T21:22:45.188 INFO:teuthology.orchestra.run.smithi052.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.371983.log.gz 2024-05-04T21:22:45.188 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.93922.log.gz 2024-05-04T21:22:45.188 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34924.log.gz 2024-05-04T21:22:45.188 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.178459.log.gz 2024-05-04T21:22:45.191 INFO:teuthology.orchestra.run.smithi052.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36970.log.gz 2024-05-04T21:22:45.191 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35349.log.gz 2024-05-04T21:22:45.192 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.482996.log.gz 2024-05-04T21:22:45.192 INFO:teuthology.orchestra.run.smithi052.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.359345.log.gz 2024-05-04T21:22:45.192 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.286960.log.gz 2024-05-04T21:22:45.192 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.226441.log.gz 2024-05-04T21:22:45.192 INFO:teuthology.orchestra.run.smithi052.stderr: 83.1% -- replaced with /var/log/ceph/ceph-client.admin.216074.log.gz 2024-05-04T21:22:45.192 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.304783.log.gz 2024-05-04T21:22:45.193 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.242358.log.gz 2024-05-04T21:22:45.193 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.153908.log.gz 2024-05-04T21:22:45.193 INFO:teuthology.orchestra.run.smithi052.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.108028.log.gz 2024-05-04T21:22:45.193 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33942.log.gz 2024-05-04T21:22:45.193 INFO:teuthology.orchestra.run.smithi052.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.232164.log.gz 2024-05-04T21:22:45.194 INFO:teuthology.orchestra.run.smithi052.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36351.log.gz 2024-05-04T21:22:45.434 INFO:teuthology.orchestra.run.smithi052.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36559.log.gz 2024-05-04T21:22:45.448 INFO:teuthology.orchestra.run.smithi052.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.328322.log.gz 2024-05-04T21:22:45.448 INFO:teuthology.orchestra.run.smithi052.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.161603.log.gz 2024-05-04T21:22:45.448 INFO:teuthology.orchestra.run.smithi052.stderr: 80.1% -- replaced with /var/log/ceph/ceph-client.admin.162443.log.gz 2024-05-04T21:22:45.449 INFO:teuthology.orchestra.run.smithi052.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.433668.log.gz 2024-05-04T21:22:45.449 INFO:teuthology.orchestra.run.smithi052.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.368670.log.gz 2024-05-04T21:22:45.449 INFO:teuthology.orchestra.run.smithi052.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.320006.log.gz 2024-05-04T21:22:45.450 INFO:teuthology.orchestra.run.smithi052.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.203799.log.gz 2024-05-04T21:22:45.451 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.170672.log.gz 2024-05-04T21:22:45.454 INFO:teuthology.orchestra.run.smithi052.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.359442.log.gz 2024-05-04T21:22:45.478 INFO:teuthology.orchestra.run.smithi052.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.345024.log.gz 2024-05-04T21:22:45.493 INFO:teuthology.orchestra.run.smithi052.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.326030.log.gz 2024-05-04T21:22:45.499 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.214803.log.gz 2024-05-04T21:22:45.515 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.378146.log.gz 2024-05-04T21:22:45.517 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.362468.log.gz 2024-05-04T21:22:45.517 INFO:teuthology.orchestra.run.smithi052.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.306410.log.gz 2024-05-04T21:22:45.538 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.396130.log.gz 2024-05-04T21:22:45.539 INFO:teuthology.orchestra.run.smithi052.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.346280.log.gz 2024-05-04T21:22:45.568 INFO:teuthology.orchestra.run.smithi052.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.437427.log.gz 2024-05-04T21:22:45.573 INFO:teuthology.orchestra.run.smithi052.stderr: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.471052.log.gz 2024-05-04T21:22:45.616 INFO:teuthology.orchestra.run.smithi052.stderr: 79.9% -- replaced with /var/log/ceph/ceph-client.admin.86652.log.gz 2024-05-04T21:22:45.617 INFO:teuthology.orchestra.run.smithi052.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.318772.log.gz 2024-05-04T21:22:45.618 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.287366.log.gz 2024-05-04T21:22:45.618 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.313648.log.gz 2024-05-04T21:22:45.638 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.463332.log.gz 2024-05-04T21:22:45.642 INFO:teuthology.orchestra.run.smithi052.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34742.log.gz 2024-05-04T21:22:45.657 INFO:teuthology.orchestra.run.smithi052.stderr: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.401730.log.gz 2024-05-04T21:22:45.657 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.179771.log.gz 2024-05-04T21:22:45.658 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.187358.log.gz 2024-05-04T21:22:45.699 INFO:teuthology.orchestra.run.smithi052.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.349055.log.gz 2024-05-04T21:22:45.700 INFO:teuthology.orchestra.run.smithi052.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.170262.log.gz 2024-05-04T21:22:45.706 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.450885.log.gz 2024-05-04T21:22:45.709 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.190244.log.gz 2024-05-04T21:22:45.713 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.150485.log.gz 2024-05-04T21:22:45.721 INFO:teuthology.orchestra.run.smithi052.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.92248.log.gz 2024-05-04T21:22:45.726 INFO:teuthology.orchestra.run.smithi052.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.288997.log.gz 2024-05-04T21:22:45.726 INFO:teuthology.orchestra.run.smithi052.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.289807.log.gz 2024-05-04T21:22:45.737 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.317478.log.gz 2024-05-04T21:22:45.738 INFO:teuthology.orchestra.run.smithi052.stderr: 94.6% -- replaced with /var/log/ceph/ceph-client.admin.328331.log.gz 2024-05-04T21:22:45.738 INFO:teuthology.orchestra.run.smithi052.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.admin.384925.log.gz 2024-05-04T21:22:45.740 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.109306.log.gz 2024-05-04T21:22:45.743 INFO:teuthology.orchestra.run.smithi052.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.465026.log.gz 2024-05-04T21:22:45.769 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.156843.log.gz 2024-05-04T21:22:45.791 INFO:teuthology.orchestra.run.smithi052.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36637.log.gz 2024-05-04T21:22:45.792 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35929.log.gz 2024-05-04T21:22:45.792 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37022.log.gz 2024-05-04T21:22:45.802 INFO:teuthology.orchestra.run.smithi052.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.316644.log.gz 2024-05-04T21:22:45.803 INFO:teuthology.orchestra.run.smithi052.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.321233.log.gz 2024-05-04T21:22:45.804 INFO:teuthology.orchestra.run.smithi052.stderr: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.410991.log.gz 2024-05-04T21:22:45.804 INFO:teuthology.orchestra.run.smithi052.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.282048.log.gz 2024-05-04T21:22:45.804 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.284496.log.gz 2024-05-04T21:22:45.805 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.430816.log.gz 2024-05-04T21:22:45.805 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.408893.log.gz 2024-05-04T21:22:45.818 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.381302.log.gz 2024-05-04T21:22:45.819 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.184466.log.gz 2024-05-04T21:22:45.821 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.456576.log.gz 2024-05-04T21:22:45.821 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.405537.log.gz 2024-05-04T21:22:45.840 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.284505.log.gz 2024-05-04T21:22:45.840 INFO:teuthology.orchestra.run.smithi052.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.157655.log.gz 2024-05-04T21:22:45.855 INFO:teuthology.orchestra.run.smithi052.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.118059.log.gz 2024-05-04T21:22:45.865 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.214397.log.gz 2024-05-04T21:22:45.870 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.362048.log.gz 2024-05-04T21:22:45.878 INFO:teuthology.orchestra.run.smithi052.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.281231.log.gz 2024-05-04T21:22:45.888 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.293617.log.gz 2024-05-04T21:22:45.894 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.100961.log.gz 2024-05-04T21:22:45.947 INFO:teuthology.orchestra.run.smithi052.stderr: 94.8% -- replaced with /var/log/ceph/ceph-client.admin.128852.log.gz 2024-05-04T21:22:45.953 INFO:teuthology.orchestra.run.smithi052.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.319186.log.gz 2024-05-04T21:22:45.953 INFO:teuthology.orchestra.run.smithi052.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.287762.log.gz 2024-05-04T21:22:45.953 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.312702.log.gz 2024-05-04T21:22:46.008 INFO:teuthology.orchestra.run.smithi052.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.125885.log.gz 2024-05-04T21:22:46.009 INFO:teuthology.orchestra.run.smithi052.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.481270.log.gz 2024-05-04T21:22:46.013 INFO:teuthology.orchestra.run.smithi052.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.482127.log.gz 2024-05-04T21:22:46.014 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.376083.log.gz 2024-05-04T21:22:46.017 INFO:teuthology.orchestra.run.smithi052.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.385046.log.gz 2024-05-04T21:22:46.019 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.354321.log.gz 2024-05-04T21:22:46.025 INFO:teuthology.orchestra.run.smithi052.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.306002.log.gz 2024-05-04T21:22:46.030 INFO:teuthology.orchestra.run.smithi052.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.427827.log.gz 2024-05-04T21:22:46.052 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.241960.log.gz 2024-05-04T21:22:46.055 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.199249.log.gz 2024-05-04T21:22:46.078 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.157249.log.gz 2024-05-04T21:22:46.114 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.94365.log.gz 2024-05-04T21:22:46.261 INFO:teuthology.orchestra.run.smithi052.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.163747.log.gz 2024-05-04T21:22:46.275 INFO:teuthology.orchestra.run.smithi052.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.358209.log.gz 2024-05-04T21:22:46.277 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.370321.log.gz 2024-05-04T21:22:46.278 INFO:teuthology.orchestra.run.smithi052.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.168540.log.gz 2024-05-04T21:22:46.281 INFO:teuthology.orchestra.run.smithi052.stderr: 78.6% -- replaced with /var/log/ceph/ceph-client.admin.471868.log.gz 2024-05-04T21:22:46.284 INFO:teuthology.orchestra.run.smithi052.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.239290.log.gz 2024-05-04T21:22:46.286 INFO:teuthology.orchestra.run.smithi052.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.218942.log.gz 2024-05-04T21:22:46.293 INFO:teuthology.orchestra.run.smithi052.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.462859.log.gz 2024-05-04T21:22:46.314 INFO:teuthology.orchestra.run.smithi052.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.201683.log.gz 2024-05-04T21:22:46.336 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.85839.log.gz 2024-05-04T21:22:46.338 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.375677.log.gz 2024-05-04T21:22:46.367 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36944.log.gz 2024-05-04T21:22:46.374 INFO:teuthology.orchestra.run.smithi052.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33705.log.gz 2024-05-04T21:22:46.380 INFO:teuthology.orchestra.run.smithi052.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.284090.log.gz 2024-05-04T21:22:46.380 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.355231.log.gz 2024-05-04T21:22:46.382 INFO:teuthology.orchestra.run.smithi052.stderr: 80.0% -- replaced with /var/log/ceph/ceph-client.admin.210839.log.gz 2024-05-04T21:22:46.383 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.241025.log.gz 2024-05-04T21:22:46.384 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.291469.log.gz 2024-05-04T21:22:46.388 INFO:teuthology.orchestra.run.smithi052.stderr: 79.9% -- replaced with /var/log/ceph/ceph-client.admin.222624.log.gz 2024-05-04T21:22:46.391 INFO:teuthology.orchestra.run.smithi052.stderr: 79.4% -- replaced with /var/log/ceph/ceph-client.admin.456170.log.gz 2024-05-04T21:22:46.393 INFO:teuthology.orchestra.run.smithi052.stderr: 78.5% -- replaced with /var/log/ceph/ceph-client.admin.466344.log.gz 2024-05-04T21:22:46.395 INFO:teuthology.orchestra.run.smithi052.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.488118.log.gz 2024-05-04T21:22:46.401 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.404186.log.gz 2024-05-04T21:22:46.412 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.78672.log.gz 2024-05-04T21:22:46.413 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.425618.log.gz 2024-05-04T21:22:46.437 INFO:teuthology.orchestra.run.smithi052.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.304367.log.gz 2024-05-04T21:22:46.445 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.213141.log.gz 2024-05-04T21:22:46.447 INFO:teuthology.orchestra.run.smithi052.stderr: 79.6% -- replaced with /var/log/ceph/ceph-client.admin.294920.log.gz 2024-05-04T21:22:46.459 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.173514.log.gz 2024-05-04T21:22:46.460 INFO:teuthology.orchestra.run.smithi052.stderr: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.230088.log.gz 2024-05-04T21:22:46.460 INFO:teuthology.orchestra.run.smithi052.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.230907.log.gz 2024-05-04T21:22:46.533 INFO:teuthology.orchestra.run.smithi052.stderr: 77.7% -- replaced with /var/log/ceph/ceph-client.admin.125474.log.gz 2024-05-04T21:22:46.544 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.311388.log.gz 2024-05-04T21:22:46.546 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.319609.log.gz 2024-05-04T21:22:46.648 INFO:teuthology.orchestra.run.smithi052.stderr: 78.6% -- replaced with /var/log/ceph/ceph-client.admin.296986.log.gz 2024-05-04T21:22:46.650 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.296181.log.gz 2024-05-04T21:22:46.665 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.345882.log.gz 2024-05-04T21:22:46.686 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.217727.log.gz 2024-05-04T21:22:46.692 INFO:teuthology.orchestra.run.smithi052.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.174317.log.gz 2024-05-04T21:22:46.696 INFO:teuthology.orchestra.run.smithi052.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.194705.log.gz 2024-05-04T21:22:46.699 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.120967.log.gz 2024-05-04T21:22:46.708 INFO:teuthology.orchestra.run.smithi052.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.153917.log.gz 2024-05-04T21:22:46.718 INFO:teuthology.orchestra.run.smithi052.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37104.log.gz 2024-05-04T21:22:46.727 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.188576.log.gz 2024-05-04T21:22:46.756 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37076.log.gz 2024-05-04T21:22:46.787 INFO:teuthology.orchestra.run.smithi052.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.206353.log.gz 2024-05-04T21:22:46.793 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.343762.log.gz 2024-05-04T21:22:46.795 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.177178.log.gz 2024-05-04T21:22:46.797 INFO:teuthology.orchestra.run.smithi052.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.112642.log.gz 2024-05-04T21:22:46.818 INFO:teuthology.orchestra.run.smithi052.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.325151.log.gz 2024-05-04T21:22:46.818 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.441343.log.gz 2024-05-04T21:22:46.859 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34976.log.gz 2024-05-04T21:22:46.869 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.301048.log.gz 2024-05-04T21:22:46.921 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33710.log.gz 2024-05-04T21:22:46.927 INFO:teuthology.orchestra.run.smithi052.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.81727.log.gz 2024-05-04T21:22:46.936 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.305597.log.gz 2024-05-04T21:22:46.936 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.216494.log.gz 2024-05-04T21:22:46.952 INFO:teuthology.orchestra.run.smithi052.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.470232.log.gz 2024-05-04T21:22:46.953 INFO:teuthology.orchestra.run.smithi052.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.440494.log.gz 2024-05-04T21:22:46.955 INFO:teuthology.orchestra.run.smithi052.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.327842.log.gz 2024-05-04T21:22:47.010 INFO:teuthology.orchestra.run.smithi052.stderr: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.371134.log.gz 2024-05-04T21:22:47.016 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.324279.log.gz 2024-05-04T21:22:47.031 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.165061.log.gz 2024-05-04T21:22:47.041 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.209619.log.gz 2024-05-04T21:22:47.042 INFO:teuthology.orchestra.run.smithi052.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.156428.log.gz 2024-05-04T21:22:47.044 INFO:teuthology.orchestra.run.smithi052.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.127724.log.gz 2024-05-04T21:22:47.054 INFO:teuthology.orchestra.run.smithi052.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.admin.347611.log.gz 2024-05-04T21:22:47.086 INFO:teuthology.orchestra.run.smithi052.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.361637.log.gz 2024-05-04T21:22:47.110 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35615.log.gz 2024-05-04T21:22:47.110 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35693.log.gz 2024-05-04T21:22:47.122 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.323396.log.gz 2024-05-04T21:22:47.127 INFO:teuthology.orchestra.run.smithi052.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.216896.log.gz 2024-05-04T21:22:47.127 INFO:teuthology.orchestra.run.smithi052.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.217718.log.gz 2024-05-04T21:22:47.128 INFO:teuthology.orchestra.run.smithi052.stderr: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.405528.log.gz 2024-05-04T21:22:47.134 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.484752.log.gz 2024-05-04T21:22:47.136 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.371992.log.gz 2024-05-04T21:22:47.144 INFO:teuthology.orchestra.run.smithi052.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.229283.log.gz 2024-05-04T21:22:47.154 INFO:teuthology.orchestra.run.smithi052.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.97688.log.gz 2024-05-04T21:22:47.162 INFO:teuthology.orchestra.run.smithi052.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.483453.log.gz 2024-05-04T21:22:47.184 INFO:teuthology.orchestra.run.smithi052.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.382974.log.gz 2024-05-04T21:22:47.282 INFO:teuthology.orchestra.run.smithi111.stderr: 95.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-04T21:22:47.288 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.160298.log.gz 2024-05-04T21:22:47.304 INFO:teuthology.orchestra.run.smithi052.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.105851.log.gz 2024-05-04T21:22:47.307 INFO:teuthology.orchestra.run.smithi052.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.150066.log.gz 2024-05-04T21:22:47.314 INFO:teuthology.orchestra.run.smithi052.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.150920.log.gz 2024-05-04T21:22:47.356 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.371143.log.gz 2024-05-04T21:22:47.405 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.304376.log.gz 2024-05-04T21:22:47.407 INFO:teuthology.orchestra.run.smithi052.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.371556.log.gz 2024-05-04T21:22:47.412 INFO:teuthology.orchestra.run.smithi052.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.434968.log.gz 2024-05-04T21:22:47.414 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.446021.log.gz 2024-05-04T21:22:47.418 INFO:teuthology.orchestra.run.smithi052.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.419622.log.gz 2024-05-04T21:22:47.421 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.164626.log.gz 2024-05-04T21:22:47.424 INFO:teuthology.orchestra.run.smithi052.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.163321.log.gz 2024-05-04T21:22:47.425 INFO:teuthology.orchestra.run.smithi052.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.224710.log.gz 2024-05-04T21:22:47.426 INFO:teuthology.orchestra.run.smithi052.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.473991.log.gz 2024-05-04T21:22:47.430 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34098.log.gz 2024-05-04T21:22:47.561 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.484336.log.gz 2024-05-04T21:22:47.616 INFO:teuthology.orchestra.run.smithi052.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.160733.log.gz 2024-05-04T21:22:47.634 INFO:teuthology.orchestra.run.smithi052.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.228868.log.gz 2024-05-04T21:22:47.638 INFO:teuthology.orchestra.run.smithi052.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.213982.log.gz 2024-05-04T21:22:47.651 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.480841.log.gz 2024-05-04T21:22:47.654 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.329787.log.gz 2024-05-04T21:22:47.703 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.302279.log.gz 2024-05-04T21:22:47.864 INFO:teuthology.orchestra.run.smithi052.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.400026.log.gz 2024-05-04T21:22:47.873 INFO:teuthology.orchestra.run.smithi052.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.348244.log.gz 2024-05-04T21:22:47.877 INFO:teuthology.orchestra.run.smithi052.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.80435.log.gz 2024-05-04T21:22:47.883 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.297400.log.gz 2024-05-04T21:22:48.046 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.366648.log.gz 2024-05-04T21:22:48.061 INFO:teuthology.orchestra.run.smithi052.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.328829.log.gz 2024-05-04T21:22:48.061 INFO:teuthology.orchestra.run.smithi052.stderr: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.103798.log.gz 2024-05-04T21:22:48.162 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.419075.log.gz 2024-05-04T21:22:48.169 INFO:teuthology.orchestra.run.smithi052.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.180168.log.gz 2024-05-04T21:22:48.174 INFO:teuthology.orchestra.run.smithi052.stderr: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.322461.log.gz 2024-05-04T21:22:48.178 INFO:teuthology.orchestra.run.smithi052.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.385668.log.gz 2024-05-04T21:22:48.180 INFO:teuthology.orchestra.run.smithi052.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.399209.log.gz 2024-05-04T21:22:48.187 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.231322.log.gz 2024-05-04T21:22:48.189 INFO:teuthology.orchestra.run.smithi052.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.200469.log.gz 2024-05-04T21:22:48.248 INFO:teuthology.orchestra.run.smithi052.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.410581.log.gz 2024-05-04T21:22:48.307 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.359393.log.gz 2024-05-04T21:22:48.354 INFO:teuthology.orchestra.run.smithi052.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.189392.log.gz 2024-05-04T21:22:48.377 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.102985.log.gz 2024-05-04T21:22:48.385 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.159023.log.gz 2024-05-04T21:22:48.385 INFO:teuthology.orchestra.run.smithi052.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.467208.log.gz 2024-05-04T21:22:48.414 INFO:teuthology.orchestra.run.smithi052.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.458608.log.gz 2024-05-04T21:22:48.418 INFO:teuthology.orchestra.run.smithi052.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.317921.log.gz 2024-05-04T21:22:48.423 INFO:teuthology.orchestra.run.smithi052.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.483005.log.gz 2024-05-04T21:22:48.441 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.189798.log.gz 2024-05-04T21:22:48.441 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.201284.log.gz 2024-05-04T21:22:48.450 INFO:teuthology.orchestra.run.smithi052.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.468925.log.gz 2024-05-04T21:22:48.500 INFO:teuthology.orchestra.run.smithi052.stderr: 96.0% -- replaced with /var/log/ceph/ceph-client.admin.186090.log.gz 2024-05-04T21:22:48.659 INFO:teuthology.orchestra.run.smithi052.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.344605.log.gz 2024-05-04T21:22:48.673 INFO:teuthology.orchestra.run.smithi052.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.120552.log.gz 2024-05-04T21:22:48.676 INFO:teuthology.orchestra.run.smithi052.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.175947.log.gz 2024-05-04T21:22:48.685 INFO:teuthology.orchestra.run.smithi052.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.411399.log.gz 2024-05-04T21:22:48.706 INFO:teuthology.orchestra.run.smithi052.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.296581.log.gz 2024-05-04T21:22:48.710 INFO:teuthology.orchestra.run.smithi052.stderr: 77.3% -- replaced with /var/log/ceph/ceph-client.admin.84971.log.gz 2024-05-04T21:22:48.716 INFO:teuthology.orchestra.run.smithi052.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.428295.log.gz 2024-05-04T21:22:48.717 INFO:teuthology.orchestra.run.smithi052.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.168105.log.gz 2024-05-04T21:22:48.720 INFO:teuthology.orchestra.run.smithi052.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.283273.log.gz 2024-05-04T21:22:48.723 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.110547.log.gz 2024-05-04T21:22:48.729 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.454961.log.gz 2024-05-04T21:22:48.760 INFO:teuthology.orchestra.run.smithi052.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.326866.log.gz 2024-05-04T21:22:48.776 INFO:teuthology.orchestra.run.smithi052.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35132.log.gz 2024-05-04T21:22:48.783 INFO:teuthology.orchestra.run.smithi052.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36117.log.gz 2024-05-04T21:22:48.921 INFO:teuthology.orchestra.run.smithi052.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.129509.log.gz 2024-05-04T21:22:48.923 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33493.log.gz 2024-05-04T21:22:48.926 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.405124.log.gz 2024-05-04T21:22:48.945 INFO:teuthology.orchestra.run.smithi052.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.292806.log.gz 2024-05-04T21:22:48.949 INFO:teuthology.orchestra.run.smithi052.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.359659.log.gz 2024-05-04T21:22:48.969 INFO:teuthology.orchestra.run.smithi052.stderr: 79.6% -- replaced with /var/log/ceph/ceph-client.admin.450657.log.gz 2024-05-04T21:22:48.972 INFO:teuthology.orchestra.run.smithi052.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.239709.log.gz 2024-05-04T21:22:48.974 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.470652.log.gz 2024-05-04T21:22:48.975 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.351549.log.gz 2024-05-04T21:22:48.981 INFO:teuthology.orchestra.run.smithi052.stderr: 83.3% -- replaced with /var/log/ceph/ceph-client.admin.210428.log.gz 2024-05-04T21:22:48.986 INFO:teuthology.orchestra.run.smithi052.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.2.34608.log.gz 2024-05-04T21:22:49.018 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37165.log.gz 2024-05-04T21:22:49.028 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.145039.log.gz 2024-05-04T21:22:49.219 INFO:teuthology.orchestra.run.smithi052.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.295732.log.gz 2024-05-04T21:22:49.250 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.199655.log.gz 2024-05-04T21:22:49.326 INFO:teuthology.orchestra.run.smithi052.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.301039.log.gz 2024-05-04T21:22:49.327 INFO:teuthology.orchestra.run.smithi052.stderr: 80.0% -- replaced with /var/log/ceph/ceph-client.admin.303088.log.gz 2024-05-04T21:22:49.370 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.434519.log.gz 2024-05-04T21:22:49.459 INFO:teuthology.orchestra.run.smithi052.stderr: 83.1% -- replaced with /var/log/ceph/ceph-client.admin.370723.log.gz 2024-05-04T21:22:49.462 INFO:teuthology.orchestra.run.smithi052.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.198790.log.gz 2024-05-04T21:22:49.474 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.295335.log.gz 2024-05-04T21:22:49.490 INFO:teuthology.orchestra.run.smithi052.stderr: 96.6% -- replaced with /var/log/ceph/ceph-client.admin.474835.log.gz 2024-05-04T21:22:49.646 INFO:teuthology.orchestra.run.smithi052.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.128545.log.gz 2024-05-04T21:22:49.665 INFO:teuthology.orchestra.run.smithi052.stderr: 79.7% -- replaced with /var/log/ceph/ceph-client.admin.129500.log.gz 2024-05-04T21:22:49.667 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.350739.log.gz 2024-05-04T21:22:49.671 INFO:teuthology.orchestra.run.smithi052.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.105031.log.gz 2024-05-04T21:22:49.676 INFO:teuthology.orchestra.run.smithi052.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.416406.log.gz 2024-05-04T21:22:49.677 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.310982.log.gz 2024-05-04T21:22:49.685 INFO:teuthology.orchestra.run.smithi052.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.463729.log.gz 2024-05-04T21:22:49.727 INFO:teuthology.orchestra.run.smithi052.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.418186.log.gz 2024-05-04T21:22:49.729 INFO:teuthology.orchestra.run.smithi052.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.153037.log.gz 2024-05-04T21:22:49.759 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33212.log.gz 2024-05-04T21:22:49.972 INFO:teuthology.orchestra.run.smithi052.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.184863.log.gz 2024-05-04T21:22:49.976 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.409299.log.gz 2024-05-04T21:22:49.983 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.231728.log.gz 2024-05-04T21:22:49.997 INFO:teuthology.orchestra.run.smithi052.stderr: 78.5% -- replaced with /var/log/ceph/ceph-client.admin.170663.log.gz 2024-05-04T21:22:50.007 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.225629.log.gz 2024-05-04T21:22:50.016 INFO:teuthology.orchestra.run.smithi052.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.311794.log.gz 2024-05-04T21:22:50.046 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.107572.log.gz 2024-05-04T21:22:50.393 INFO:teuthology.orchestra.run.smithi052.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.214794.log.gz 2024-05-04T21:22:50.394 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-04T21:22:50.394 INFO:teuthology.orchestra.run.smithi052.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.359297.log.gz 2024-05-04T21:22:50.417 INFO:teuthology.orchestra.run.smithi052.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.203371.log.gz 2024-05-04T21:22:50.419 INFO:teuthology.orchestra.run.smithi052.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35956.log.gz 2024-05-04T21:22:50.421 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.222222.log.gz 2024-05-04T21:22:50.791 INFO:teuthology.orchestra.run.smithi052.stderr: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.366229.log.gz 2024-05-04T21:22:50.792 INFO:teuthology.orchestra.run.smithi052.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.380897.log.gz 2024-05-04T21:22:50.796 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.386493.log.gz 2024-05-04T21:22:50.797 INFO:teuthology.orchestra.run.smithi052.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.472339.log.gz 2024-05-04T21:22:50.801 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.414748.log.gz 2024-05-04T21:22:50.812 INFO:teuthology.orchestra.run.smithi052.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.384195.log.gz 2024-05-04T21:22:50.986 INFO:teuthology.orchestra.run.smithi052.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.229690.log.gz 2024-05-04T21:22:50.992 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.357812.log.gz 2024-05-04T21:22:50.993 INFO:teuthology.orchestra.run.smithi052.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.116759.log.gz 2024-05-04T21:22:50.996 INFO:teuthology.orchestra.run.smithi052.stderr: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.436727.log.gz 2024-05-04T21:22:51.001 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.458211.log.gz 2024-05-04T21:22:51.009 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.323866.log.gz 2024-05-04T21:22:51.258 INFO:teuthology.orchestra.run.smithi052.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.299012.log.gz 2024-05-04T21:22:51.267 INFO:teuthology.orchestra.run.smithi052.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.429992.log.gz 2024-05-04T21:22:51.272 INFO:teuthology.orchestra.run.smithi052.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.357368.log.gz 2024-05-04T21:22:51.280 INFO:teuthology.orchestra.run.smithi052.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.110120.log.gz 2024-05-04T21:22:51.412 INFO:teuthology.orchestra.run.smithi052.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.330709.log.gz 2024-05-04T21:22:51.417 INFO:teuthology.orchestra.run.smithi052.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.126392.log.gz 2024-05-04T21:22:51.484 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36455.log.gz 2024-05-04T21:22:51.661 INFO:teuthology.orchestra.run.smithi052.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.359384.log.gz 2024-05-04T21:22:51.665 INFO:teuthology.orchestra.run.smithi052.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.457805.log.gz 2024-05-04T21:22:51.665 INFO:teuthology.orchestra.run.smithi052.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.476553.log.gz 2024-05-04T21:22:51.828 INFO:teuthology.orchestra.run.smithi052.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.91510.log.gz 2024-05-04T21:22:51.831 INFO:teuthology.orchestra.run.smithi052.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33499.log.gz 2024-05-04T21:22:52.036 INFO:teuthology.orchestra.run.smithi052.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.346289.log.gz 2024-05-04T21:22:52.613 INFO:teuthology.orchestra.run.smithi052.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.282871.log.gz 2024-05-04T21:22:52.622 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.368679.log.gz 2024-05-04T21:22:52.624 INFO:teuthology.orchestra.run.smithi052.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.240122.log.gz 2024-05-04T21:22:53.110 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.299831.log.gz 2024-05-04T21:22:53.334 INFO:teuthology.orchestra.run.smithi052.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.155276.log.gz 2024-05-04T21:22:54.157 INFO:teuthology.orchestra.run.smithi052.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.154805.log.gz 2024-05-04T21:22:54.218 INFO:teuthology.orchestra.run.smithi052.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.313163.log.gz 2024-05-04T21:22:54.259 INFO:teuthology.orchestra.run.smithi052.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.78077.log.gz 2024-05-04T21:22:57.136 INFO:teuthology.orchestra.run.smithi052.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34228.log.gz 2024-05-04T21:22:58.782 INFO:teuthology.orchestra.run.smithi052.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.330257.log.gz 2024-05-04T21:23:01.428 INFO:teuthology.orchestra.run.smithi052.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.91519.log.gz 2024-05-04T21:23:01.792 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.192936.log.gz 2024-05-04T21:23:03.273 INFO:teuthology.orchestra.run.smithi052.stderr: 94.6% -- replaced with /var/log/ceph/ceph-client.admin.79496.log.gz 2024-05-04T21:23:04.735 INFO:teuthology.orchestra.run.smithi052.stderr: 95.9% -- replaced with /var/log/ceph/ceph-client.admin.224274.log.gz 2024-05-04T21:23:13.665 INFO:teuthology.orchestra.run.smithi052.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.212549.log.gz 2024-05-04T21:23:13.685 INFO:teuthology.orchestra.run.smithi052.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.admin.354736.log.gz 2024-05-04T21:23:14.967 INFO:teuthology.orchestra.run.smithi052.stderr: 95.8% -- replaced with /var/log/ceph/ceph-client.admin.190711.log.gz 2024-05-04T21:23:16.229 INFO:teuthology.orchestra.run.smithi052.stderr: 94.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-04T21:23:16.668 INFO:teuthology.orchestra.run.smithi052.stderr: 97.2% -- replaced with /var/log/ceph/ceph-client.admin.347132.log.gz 2024-05-04T21:23:16.803 INFO:teuthology.orchestra.run.smithi052.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.465035.log.gz 2024-05-04T21:23:23.482 INFO:teuthology.orchestra.run.smithi052.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.108445.log.gz 2024-05-04T21:23:25.521 INFO:teuthology.orchestra.run.smithi052.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.486426.log.gz 2024-05-04T21:23:25.555 INFO:teuthology.orchestra.run.smithi052.stderr: 94.8% -- replaced with /var/log/ceph/ceph-client.admin.93931.log.gz 2024-05-04T21:23:26.793 INFO:teuthology.orchestra.run.smithi052.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.211660.log.gz 2024-05-04T21:23:27.256 INFO:teuthology.orchestra.run.smithi052.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.464587.log.gz 2024-05-04T21:23:27.456 INFO:teuthology.orchestra.run.smithi052.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.414190.log.gz 2024-05-04T21:23:28.018 INFO:teuthology.orchestra.run.smithi052.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.193445.log.gz 2024-05-04T21:23:29.880 INFO:teuthology.orchestra.run.smithi052.stderr: 97.3% 97.1% -- replaced with /var/log/ceph/ceph-client.admin.291972.log.gz -- replaced with /var/log/ceph/ceph-client.admin.291478.log.gz 2024-05-04T21:23:29.880 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-04T21:23:30.067 INFO:teuthology.orchestra.run.smithi052.stderr: 96.9% -- replaced with /var/log/ceph/ceph-client.admin.394051.log.gz 2024-05-04T21:23:30.348 INFO:teuthology.orchestra.run.smithi052.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.80893.log.gz 2024-05-04T21:23:30.349 INFO:teuthology.orchestra.run.smithi052.stderr: 97.2% -- replaced with /var/log/ceph/ceph-client.admin.81311.log.gz 2024-05-04T21:23:31.712 INFO:teuthology.orchestra.run.smithi052.stderr: 97.2% -- replaced with /var/log/ceph/ceph-client.admin.465916.log.gz 2024-05-04T21:23:31.712 INFO:teuthology.orchestra.run.smithi052.stderr: 97.0% -- replaced with /var/log/ceph/ceph-client.admin.107126.log.gz 2024-05-04T21:23:32.780 INFO:teuthology.orchestra.run.smithi052.stderr: 97.0% -- replaced with /var/log/ceph/ceph-client.admin.112225.log.gz 2024-05-04T21:23:34.958 INFO:teuthology.orchestra.run.smithi052.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.admin.107581.log.gz 2024-05-04T21:23:35.227 INFO:teuthology.orchestra.run.smithi052.stderr: 96.9% -- replaced with /var/log/ceph/ceph-client.admin.91057.log.gz 2024-05-04T21:23:35.422 INFO:teuthology.orchestra.run.smithi052.stderr: 97.2% -- replaced with /var/log/ceph/ceph-client.admin.363717.log.gz 2024-05-04T21:23:35.486 INFO:teuthology.orchestra.run.smithi052.stderr: 96.9% -- replaced with /var/log/ceph/ceph-client.admin.85386.log.gz 2024-05-04T21:23:35.544 INFO:teuthology.orchestra.run.smithi052.stderr: 97.2% -- replaced with /var/log/ceph/ceph-client.admin.316213.log.gz 2024-05-04T21:23:35.682 INFO:teuthology.orchestra.run.smithi052.stderr: 97.2% -- replaced with /var/log/ceph/ceph-client.admin.322940.log.gz 2024-05-04T21:23:36.886 INFO:teuthology.orchestra.run.smithi052.stderr: 97.2% -- replaced with /var/log/ceph/ceph-client.admin.241518.log.gz 2024-05-04T21:23:37.626 INFO:teuthology.orchestra.run.smithi052.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.212120.log.gz 2024-05-04T21:23:37.940 INFO:teuthology.orchestra.run.smithi052.stderr: 95.9% -- replaced with /var/log/ceph/ceph-client.admin.189807.log.gz 2024-05-04T21:23:39.087 INFO:teuthology.orchestra.run.smithi052.stderr: 97.4% -- replaced with /var/log/ceph/ceph-client.admin.291050.log.gz 2024-05-04T21:23:39.623 INFO:teuthology.orchestra.run.smithi052.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.465478.log.gz 2024-05-04T21:23:39.960 INFO:teuthology.orchestra.run.smithi052.stderr: 97.0% -- replaced with /var/log/ceph/ceph-client.admin.106262.log.gz 2024-05-04T21:23:40.672 INFO:teuthology.orchestra.run.smithi052.stderr: 96.9% -- replaced with /var/log/ceph/ceph-client.admin.90596.log.gz 2024-05-04T21:23:41.850 INFO:teuthology.orchestra.run.smithi052.stderr: 97.2% -- replaced with /var/log/ceph/ceph-client.admin.82594.log.gz 2024-05-04T21:23:44.009 INFO:teuthology.orchestra.run.smithi052.stderr: 96.3% -- replaced with /var/log/ceph/ceph-client.admin.353634.log.gz 2024-05-04T21:23:51.198 INFO:teuthology.orchestra.run.smithi052.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.209180.log.gz 2024-05-04T21:23:56.771 INFO:teuthology.orchestra.run.smithi052.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.436193.log.gz 2024-05-04T21:23:59.680 INFO:teuthology.orchestra.run.smithi052.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.admin.87067.log.gz 2024-05-04T21:24:02.704 INFO:teuthology.orchestra.run.smithi052.stderr: 97.2% -- replaced with /var/log/ceph/ceph-client.admin.355637.log.gz 2024-05-04T21:24:15.406 INFO:teuthology.orchestra.run.smithi111.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-04T21:24:20.202 INFO:teuthology.orchestra.run.smithi052.stderr: 97.9% -- replaced with /var/log/ceph/ceph-client.admin.434084.log.gz 2024-05-04T21:24:20.213 INFO:teuthology.orchestra.run.smithi052.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.admin.206787.log.gz 2024-05-04T21:24:25.346 INFO:teuthology.orchestra.run.smithi111.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-04T21:24:54.372 INFO:teuthology.orchestra.run.smithi111.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-04T21:25:09.207 INFO:teuthology.orchestra.run.smithi111.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-04T21:25:20.968 INFO:teuthology.orchestra.run.smithi052.stderr: 97.5% -- replaced with /var/log/ceph/ceph-client.admin.97252.log.gz 2024-05-04T21:25:21.593 INFO:teuthology.orchestra.run.smithi052.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.215210.log.gz 2024-05-04T21:25:41.716 INFO:teuthology.orchestra.run.smithi052.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.84338.log.gz 2024-05-04T21:25:51.339 INFO:teuthology.orchestra.run.smithi052.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.148774.log.gz 2024-05-04T21:26:13.537 INFO:teuthology.orchestra.run.smithi052.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.116768.log.gz 2024-05-04T21:26:24.344 INFO:teuthology.orchestra.run.smithi052.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.427837.log.gz 2024-05-04T21:26:24.875 INFO:teuthology.orchestra.run.smithi052.stderr: 96.3% -- replaced with /var/log/ceph/ceph-client.admin.190253.log.gz 2024-05-04T21:26:39.447 INFO:teuthology.orchestra.run.smithi052.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.admin.308467.log.gz 2024-05-04T21:26:55.082 INFO:teuthology.orchestra.run.smithi111.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-04T21:27:06.859 INFO:teuthology.orchestra.run.smithi111.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-04T21:27:06.861 INFO:teuthology.orchestra.run.smithi111.stderr: 2024-05-04T21:27:06.861 INFO:teuthology.orchestra.run.smithi111.stderr:real 4m26.666s 2024-05-04T21:27:06.861 INFO:teuthology.orchestra.run.smithi111.stderr:user 6m29.888s 2024-05-04T21:27:06.861 INFO:teuthology.orchestra.run.smithi111.stderr:sys 0m18.518s 2024-05-04T21:27:29.240 INFO:teuthology.orchestra.run.smithi052.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.149243.log.gz 2024-05-04T21:29:49.048 INFO:teuthology.orchestra.run.smithi052.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-04T21:29:57.676 INFO:teuthology.orchestra.run.smithi052.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.325567.log.gz 2024-05-04T21:30:14.354 INFO:teuthology.orchestra.run.smithi052.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.admin.125894.log.gz 2024-05-04T21:31:02.739 INFO:teuthology.orchestra.run.smithi052.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.111363.log.gz 2024-05-04T21:31:17.012 INFO:teuthology.orchestra.run.smithi052.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.79989.log.gz 2024-05-04T21:32:24.280 INFO:teuthology.orchestra.run.smithi052.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.admin.158507.log.gz 2024-05-04T21:32:42.196 INFO:teuthology.orchestra.run.smithi052.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.129944.log.gz 2024-05-04T21:34:14.801 INFO:teuthology.orchestra.run.smithi052.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.356507.log.gz 2024-05-04T21:34:18.482 INFO:teuthology.orchestra.run.smithi052.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.82134.log.gz 2024-05-04T21:34:29.071 INFO:teuthology.orchestra.run.smithi052.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.427341.log.gz 2024-05-04T21:37:15.005 INFO:teuthology.orchestra.run.smithi052.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.admin.486865.log.gz 2024-05-04T21:39:19.003 INFO:teuthology.orchestra.run.smithi052.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.450666.log.gz 2024-05-04T21:40:12.406 INFO:teuthology.orchestra.run.smithi052.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.admin.224719.log.gz 2024-05-04T21:43:06.022 INFO:teuthology.orchestra.run.smithi052.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.80444.log.gz 2024-05-04T21:44:23.957 INFO:teuthology.orchestra.run.smithi052.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.admin.332140.log.gz 2024-05-04T21:44:38.911 INFO:teuthology.orchestra.run.smithi052.stderr: 90.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-04T21:45:49.346 INFO:teuthology.orchestra.run.smithi052.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.83053.log.gz 2024-05-04T21:46:00.781 INFO:teuthology.orchestra.run.smithi052.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.204219.log.gz 2024-05-04T21:48:09.863 INFO:teuthology.orchestra.run.smithi052.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.445321.log.gz 2024-05-04T21:48:26.617 INFO:teuthology.orchestra.run.smithi052.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.449047.log.gz 2024-05-04T21:48:35.442 INFO:teuthology.orchestra.run.smithi052.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.449810.log.gz 2024-05-04T21:48:36.581 INFO:teuthology.orchestra.run.smithi052.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.443812.log.gz 2024-05-04T21:49:24.446 INFO:teuthology.orchestra.run.smithi052.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.448328.log.gz 2024-05-04T21:49:45.299 INFO:teuthology.orchestra.run.smithi052.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.444599.log.gz 2024-05-04T21:49:45.474 INFO:teuthology.orchestra.run.smithi052.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.446030.log.gz 2024-05-04T21:50:05.787 INFO:teuthology.orchestra.run.smithi052.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.441780.log.gz 2024-05-04T21:50:44.328 INFO:teuthology.orchestra.run.smithi052.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.446778.log.gz 2024-05-04T21:51:00.125 INFO:teuthology.orchestra.run.smithi052.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.447596.log.gz 2024-05-04T21:54:24.253 INFO:teuthology.orchestra.run.smithi052.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.admin.244412.log.gz 2024-05-04T21:59:12.507 INFO:teuthology.orchestra.run.smithi052.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.admin.155805.log.gz 2024-05-04T22:00:09.118 INFO:teuthology.orchestra.run.smithi052.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.386899.log.gz 2024-05-04T22:04:42.664 INFO:teuthology.orchestra.run.smithi052.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.234699.log.gz 2024-05-04T22:08:14.122 INFO:teuthology.orchestra.run.smithi052.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-04T22:16:46.483 INFO:teuthology.orchestra.run.smithi052.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.378557.log.gz 2024-05-04T22:16:50.324 INFO:teuthology.orchestra.run.smithi052.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.379583.log.gz 2024-05-04T22:17:58.348 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-04T22:23:28.742 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-04T22:26:23.195 INFO:teuthology.orchestra.run.smithi052.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-04T22:27:01.513 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-04T22:27:04.173 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.122737.log.gz 2024-05-04T22:54:02.960 INFO:teuthology.orchestra.run.smithi052.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.admin.113063.log.gz 2024-05-04T23:10:08.456 INFO:teuthology.orchestra.run.smithi052.stderr: 97.0% -- replaced with /var/log/ceph/ceph-client.admin.489030.log.gz 2024-05-04T23:13:00.420 INFO:teuthology.orchestra.run.smithi052.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.461105.log.gz 2024-05-04T23:13:00.421 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-04T23:13:00.421 INFO:teuthology.orchestra.run.smithi052.stderr:real 110m20.240s 2024-05-04T23:13:00.421 INFO:teuthology.orchestra.run.smithi052.stderr:user 88m25.404s 2024-05-04T23:13:00.421 INFO:teuthology.orchestra.run.smithi052.stderr:sys 4m23.695s 2024-05-04T23:13:00.435 INFO:tasks.ceph:Archiving logs... 2024-05-04T23:13:00.437 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278/remote/smithi052/log 2024-05-04T23:13:00.438 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-04T23:21:54.724 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278/remote/smithi111/log 2024-05-04T23:21:54.726 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-04T23:24:40.725 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-04T23:24:40.735 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-04T23:24:40.736 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-04T23:24:40.739 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-04T23:24:40.906 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-04T23:24:40.915 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-04T23:24:40.916 DEBUG:teuthology.orchestra.run.smithi052:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T23:24:40.918 DEBUG:teuthology.orchestra.run.smithi111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T23:24:40.998 INFO:teuthology.orchestra.run.smithi111.stdout: remote refid st t when poll reach delay offset jitter 2024-05-04T23:24:40.998 INFO:teuthology.orchestra.run.smithi111.stdout:============================================================================== 2024-05-04T23:24:40.998 INFO:teuthology.orchestra.run.smithi111.stdout:+hv01.front.sepi 67.205.162.81 3 u 1025 1024 377 0.253 -4.907 2.579 2024-05-04T23:24:40.998 INFO:teuthology.orchestra.run.smithi111.stdout:+hv02.front.sepi 63.231.80.2 3 u 13 1024 377 0.225 -3.222 2.203 2024-05-04T23:24:40.999 INFO:teuthology.orchestra.run.smithi111.stdout:*hv03.front.sepi 74.6.168.72 3 u 455 1024 377 0.209 -3.061 2.197 2024-05-04T23:24:40.999 INFO:teuthology.orchestra.run.smithi111.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-05-04T23:24:41.044 INFO:teuthology.orchestra.run.smithi052.stdout: remote refid st t when poll reach delay offset jitter 2024-05-04T23:24:41.044 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================== 2024-05-04T23:24:41.044 INFO:teuthology.orchestra.run.smithi052.stdout:+hv01.front.sepi 67.205.162.81 3 u 610 1024 377 0.192 -2.928 1.716 2024-05-04T23:24:41.044 INFO:teuthology.orchestra.run.smithi052.stdout:*hv02.front.sepi 63.231.80.2 3 u 682 1024 377 0.229 -0.733 1.362 2024-05-04T23:24:41.044 INFO:teuthology.orchestra.run.smithi052.stdout:+hv03.front.sepi 74.6.168.72 3 u 553 1024 377 0.186 -0.655 0.714 2024-05-04T23:24:41.044 INFO:teuthology.orchestra.run.smithi052.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-05-04T23:24:41.045 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-04T23:24:41.057 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-04T23:24:41.058 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-04T23:24:41.067 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-04T23:24:41.076 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-04T23:24:41.119 INFO:teuthology.task.internal:Duration was 40636.525909 seconds 2024-05-04T23:24:41.119 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-04T23:24:41.128 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-04T23:24:41.128 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-04T23:24:41.131 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-04T23:24:41.505 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-04T23:24:41.505 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi052.front.sepia.ceph.com 2024-05-04T23:24:41.505 DEBUG:teuthology.orchestra.run.smithi052:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-04T23:24:41.576 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi111.front.sepia.ceph.com 2024-05-04T23:24:41.576 DEBUG:teuthology.orchestra.run.smithi111:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-04T23:24:41.597 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-04T23:24:41.597 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-04T23:24:41.599 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-04T23:24:41.764 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-04T23:24:41.765 DEBUG:teuthology.orchestra.run.smithi052:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-04T23:24:41.767 DEBUG:teuthology.orchestra.run.smithi111:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-04T23:24:42.507 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-04T23:24:42.518 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-04T23:24:42.518 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-04T23:24:42.552 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-04T23:24:42.613 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-04T23:24:42.623 DEBUG:teuthology.orchestra.run.smithi052:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-04T23:24:42.625 DEBUG:teuthology.orchestra.run.smithi111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-04T23:24:42.661 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = core 2024-05-04T23:24:42.662 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = core 2024-05-04T23:24:42.700 DEBUG:teuthology.orchestra.run.smithi052:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-04T23:24:42.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T23:24:42.746 DEBUG:teuthology.orchestra.run.smithi111:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-04T23:24:42.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T23:24:42.751 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-04T23:24:42.761 INFO:teuthology.task.internal:Transferring archived files... 2024-05-04T23:24:42.762 DEBUG:teuthology.misc:Transferring archived files from smithi052:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278/remote/smithi052 2024-05-04T23:24:42.762 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-04T23:24:42.826 DEBUG:teuthology.misc:Transferring archived files from smithi111:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278/remote/smithi111 2024-05-04T23:24:42.827 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-04T23:24:42.848 INFO:teuthology.task.internal:Removing archive directory... 2024-05-04T23:24:42.848 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-04T23:24:42.867 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-04T23:24:42.890 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-04T23:24:42.901 INFO:teuthology.task.internal:Not uploading archives. 2024-05-04T23:24:42.901 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-04T23:24:42.911 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-04T23:24:42.911 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-04T23:24:42.915 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-04T23:24:42.918 INFO:teuthology.orchestra.run.smithi052.stdout: 658639 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 4 23:24 /home/ubuntu/cephtest 2024-05-04T23:24:42.939 INFO:teuthology.orchestra.run.smithi111.stdout: 658642 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 4 23:24 /home/ubuntu/cephtest 2024-05-04T23:24:42.945 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-04T23:24:42.955 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-04T23:24:42.974 INFO:teuthology.nuke:Checking targets against current locks 2024-05-04T23:24:42.999 DEBUG:teuthology.nuke:shortname: smithi052 2024-05-04T23:24:43.000 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-04T23:24:43.023 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-04 11:58:19.653799', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDPZERTOJNDZxL2XAJILk+M8iDgy2uK9e+uQScedQDF0HhdKaCNtClQgSVnFoUD/Cy1/bMoA8+vv4Y39DyglmFA='} 2024-05-04T23:24:43.026 DEBUG:teuthology.nuke:shortname: smithi111 2024-05-04T23:24:43.026 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-04T23:24:43.050 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-05-02_19:10:09-fs-wip-yuri4-testing-2024-04-29-0642-distro-default-smithi/7686278', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-04 11:58:19.652635', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM9FpB/ROanvtTByjSaVIgOKZcrhO6T9+F6tFHY31AHl+5ybolof2Jmzat+7DOV3/ui3LkwWN3sRAFnTRfk2HaY='} 2024-05-04T23:24:43.080 INFO:teuthology.orchestra.console.smithi052:Power off 2024-05-04T23:24:43.080 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-04T23:24:43.106 DEBUG:teuthology.orchestra.console.smithi052:power off output: Chassis Power Control: Down/Off 2024-05-04T23:24:43.106 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-04T23:24:43.124 INFO:teuthology.orchestra.console.smithi111:Power off 2024-05-04T23:24:43.124 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-04T23:24:43.135 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-05-04T23:24:43.148 DEBUG:teuthology.orchestra.console.smithi111:power off output: Chassis Power Control: Down/Off 2024-05-04T23:24:43.148 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-04T23:24:43.174 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2024-05-04T23:24:47.136 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-04T23:24:47.176 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-04T23:24:47.247 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-05-04T23:24:47.287 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2024-05-04T23:24:51.249 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-04T23:24:51.288 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-04T23:24:51.360 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-05-04T23:24:51.397 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is off 2024-05-04T23:24:51.498 INFO:teuthology.orchestra.console.smithi111:Power off completed 2024-05-04T23:24:55.361 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-04T23:24:55.473 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is off 2024-05-04T23:24:55.574 INFO:teuthology.orchestra.console.smithi052:Power off completed 2024-05-04T23:24:55.679 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev} duration: 40636.52590942383 failure_reason: 'Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev)' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=eaaa91d56cfa4917ad2e983a0a07bcfc status: fail success: false 2024-05-04T23:24:55.679 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-04T23:24:55.769 INFO:teuthology.run:FAIL